HADOOP cluster some issue for installation

给namenode搭建了HA,然后根据网上的配置也配置了secondary namenode,

但是一直没有从日志中看到启动secondnary namenode,当然进程也没有。

找了很多资料,按照资料配置了,执行

hdfs getconf –secondaryNameNodes

Incorrect configuration: secondary namenode address dfs.namenode.secondary.http-address is not configured.

但其实在hdfs-site.xml中已经正确配置了该参数。

执行

hdfs secondarynamenode –format

报如下错误:

15/08/01 21:25:56 FATAL namenode.SecondaryNameNode: Failed to start secondary namenode
java.io.IOException: Cannot use SecondaryNameNode in an HA cluster. The Standby Namenode will perform checkpointing.

这个错误提示让我恍然大悟,在HA情况下,就不需要再做secondary namenode来做了,standby namendoe已经做了Secondary namenode做的事了。

 

还有一个错误就是一直报警告,本地库没有被加载,我已经编译并复制到native文件夹,还是一直报这个错误,

解决方法:启动export HADOOP_ROOT_LOGGER=DEBUG,console

随便执行一个命令如:hdfs dfs –ls /

查看日志,即可发现具体的错误,根据错误提示进行修改,一般是路径有问题或是版本不对。

时间: 2024-10-11 23:00:12

HADOOP cluster some issue for installation的相关文章

Enable Kerberos secured Hadoop cluster with Cloudera Manager

I created an secured Hadoop cluster for P&G with cloudera manager, and this document is to record how to enable kerberos secured cluster with cloudera manager. Firstly we should have a cluster that contains kerberos KDC and kerberos clients 1. Instal

Hadoop Cluster 安装

本篇源自Hadoop官网,先将中文翻译如下. 目标 本文章主要是描述如何安装和配置几个节点的Hadoop clusters,甚至于数以千计的节点数.为了了解详细的安装步骤,需要先了解如何安装在单台机器上. 本文档不包含高级的设置点,比如:安全性或者高可用性. 准备 需要安装Java环境 从Apache网站下载一个稳定的Hadoop安装镜像 安装 搭建一个Hadoop集群需要将软件安装到集群中的所有机器中,或者一个适合当前操作系统集成的系统.这样做的目标就是区分不同的硬件安装不同的功能. 典型的安

Kerberos How to Kerberize an Hadoop Cluster

Most Hadoop clusters adopt Kerberos as the authentication protocol 安装 KDC 启动Kerberos 认证需要安装 KDC 服务器和必要的软件.安装KDC 的命令可以在任何机器上执行. yum -y install krb5-server krb5-lib krb5-auth-dialog krb5-workstation 接着,在集群中的其他节点上安装Kerberos client和命令 yum -y install krb5

Hadoop: Hadoop Cluster配置文件

一.环境准备 1.系统环境 2台CentOS7服务器   NameNode ResourceManager DataNode NodeManager server1 是   是 是 server2   是 是 是 2.软件环境 java-1.8.0-openjdk java-1.8.0-openjdk-devel Hadoop 2.7.3 二.Hadoop配置文件 Hadoop的配置文件: 只读的默认配置文件:core-default.xml, hdfs-default.xml, yarn-de

Hadoop MapReduce Next Generation - Setting up a Single Node Cluster

Hadoop MapReduce Next Generation - Setting up a Single Node Cluster. Purpose This document describes how to set up and configure a single-node Hadoop installation so that you can quickly perform simple operations using Hadoop MapReduce and the Hadoop

Hadoop single Node cluster

目的: 本文描述怎么安装和配置一个单结点的Hadoop,以便搭建能快速简单操作和使用Hadoop的MapReduce和Hadoop的分布式文件系统(HDFS); 先决条件: 支持的平台 GNU/Linux 可用作开发和生产平台. Hadoop 在2000台GNU/Linux 的集群上做过演示. Windows 平台也是支持的, 下面的步骤仅对linux适用. 要在Windows上安装hadoop, 查看 wiki page. 所需软件: 在linux下,所需的软件如下: 必须安装JAVA. 推荐

ubuntu从头开始搭建hadoop伪分布式环境

13年学习过一段时间的hadoop,但是工作中用到的地方比较少,有些生疏,加上现在hadoop版本也已经比较新了,所以空闲时间想继续学习一下,找到这篇文章,从头开始搭建一个hadoop环境,转过来备忘 Hadoop developers usually test their scripts and code on a pseudo-distributed environment(also known as a single node setup), which is a virtual mach

Hadoop 2.X : 分布式安装

原文: http://disi.unitn.it/~lissandrini/notes/installing-hadoop-on-ubuntu-14.html This guide is shows step by step how to set up a multi nod cluster with Hadoop and HDFS 2.4.1 on Ubuntu 14.04. It is an update, and takes many parts from previous guides

Extending sparklyr to Compute Cost for K-means on YARN Cluster with Spark ML Library

Machine and statistical learning wizards are becoming more eager to perform analysis with Spark MLlibrary if this is only possible. It’s trendy, posh, spicy and gives the feeling of doing state of the art machine learning and being up to date with th