Zookeeper + Hadoop + Hbase部署备忘

网上类似的文章很多,本文只是记录下来备忘.本文分四大步骤:

　　　　准备工作、安装zookeeper、安装hadoop、安装hbase，下面分别详细介绍：

　 一。准备工作

1. 下载 zookeeper、hadoop、hbase三个压缩包并放入/home/servers目录并解压

2. 安装JDK （略）

3. 统一集群计算机名

本次用于测试一共5台，分别命名为release-master,slave1,slave2,slave3,slave4(可使用hostname slave1类似的命令临时修改)

4. 统一hosts

　　    在以上5台服务器的hosts里都加上以下内容:

192.168.1.101 release-master
192.168.1.102 slave1
192.168.1.103 slave2
192.168.1.104 slave3
192.168.1.105 slave4

注意,如果hosts里有类似下面这样对本机进行配置的,一定要注释掉,否则RegionServer容易出错

192.168.1.101  pc-101

     二. 安装zookeeper (参考http://www.blogjava.net/hello-yun/archive/2012/05/03/377250.html)   

ZooKeeper分布式模式安装（ZooKeeper集群）也比较容易，这里说明一下基本要点。

首先要明确的是，ZooKeeper集群是一个独立的分布式协调服务集群，“独立”的含义就是说，如果想使用ZooKeeper实现分布式应用的协 调与管理，简化协调与管理，任何分布式应用都可以使用，这就要归功于Zookeeper的数据模型（Data Model）和层次命名空间（Hierarchical Namespace）结构，详细可以参考http://zookeeper.apache.org/doc/trunk/zookeeperOver.html。在设计你的分布式应用协调服务时，首要的就是考虑如何组织层次命名空间。

下面说明分布式模式的安装配置，过程如下所示：

第一步：主机名称到IP地址映射配置

ZooKeeper集群中具有两个关键的角色：Leader和Follower。集群中所有的结点作为一个整体对分布式应用提供服务，集群中每个结 点之间都互相连接，所以，在配置的ZooKeeper集群的时候，每一个结点的host到IP地址的映射都要配置上集群中其它结点的映射信息。

例如，我的ZooKeeper集群中每个结点的配置，以slave-01为例，/etc/hosts内容如下所示：　　

192.168.1.101 release-master
192.168.1.102 slave1
192.168.1.103 slave2
192.168.1.104 slave3
192.168.1.105 slave4

ZooKeeper采用一种称为Leader election的选举算法。在整个集群运行过程中，只有一个Leader，其他的都是Follower，如果ZooKeeper集群在运行过程中 Leader出了问题，系统会采用该算法重新选出一个Leader。因此，各个结点之间要能够保证互相连接，必须配置上述映射。

ZooKeeper集群启动的时候，会首先选出一个Leader，在Leader election过程中，某一个满足选举算的结点就能成为Leader。整个集群的架构可以参考http://zookeeper.apache.org/doc/trunk/zookeeperOver.html#sc_designGoals。

第二步：修改ZooKeeper配置文件和环境变量

在主服务器（release-master）上，解压缩zookeeper-3.3.4.tar.gz，修改配置文件conf/zoo.cfg，内容如下所示：

  1. tickTime=2000
dataDir=/home/data/zkdata
clientPort=2181
initLimit=5
syncLimit=2
server.1=192.168.0.101:2888:3888
server.2=192.168.0.102:2888:3888
server.3=192.168.0.103:2888:3888
server.4=192.168.0.104:2888:3888
server.5=192.168.0.105:2888:3888

上述配置内容说明，可以参考http://zookeeper.apache.org/doc/trunk/zookeeperStarted.html#sc_RunningReplicatedZooKeeper。创建配置所需目录:

mkdir -p /home/data/zkdata

修改环境变量/etc/profile,加入

export ZOOKEEPER_HOME=/export/servers/zookeeper-3.4.6
export PATH=$JAVA_HOME/bin:$PATH:$ZOOKEEPER_HOME/bin:$ZOOKEEPER_HOME/conf

　　　　保存退出. 执行:　　　　　　　

source /etc/profile

　　　　来刷新配置.　　　　　

第三步：远程复制分发安装文件和配置

上面已经在一台机器slave-01上配置完成ZooKeeper，现在可以将该配置好的安装文件远程拷贝到集群中的各个结点对应的目录下：

scp -r /home/servers/zookeeper-3.4.6/ [email protected]:/home/servers
scp   /etc/profile [email protected]:/etc/

第四步：设置myid

在我们配置的dataDir指定的目录下面，创建一个myid文件，里面内容为一个数字，用来标识当前主机，conf/zoo.cfg文件中配置的server.X中X为什么数字，则myid文件中就输入这个数字，例如：

    1. echo "1" > /home/data/zkdata/myid

按照上述进行配置即可。

第五步：启动ZooKeeper集群

在ZooKeeper集群的每个结点上，执行启动ZooKeeper服务的脚本，如下所示：

cd /home/servers/zookeeper-3.4.6/
bin/zkServer.sh start

　　以结点slave-01为例，日志如下所示：

    1. [email protected]:~/installation/zookeeper-3.3.4$ tail -500f zookeeper.out
    2. 2012-01-08 06:51:19,117 - INFO  [main:[email protected]] - Reading configuration from: /home/hadoop/installation/zookeeper-3.3.4/bin/../conf/zoo.cfg
    3. 2012-01-08 06:51:19,133 - INFO  [main:[email protected]] - Defaulting to majority quorums
    4. 2012-01-08 06:51:19,167 - INFO  [main:[email protected]] - Starting quorum peer
    5. 2012-01-08 06:51:19,227 - INFO  [main:[email protected]] - binding to port 0.0.0.0/0.0.0.0:2181
    6. 2012-01-08 06:51:19,277 - INFO  [main:[email protected]] - tickTime set to 2000
    7. 2012-01-08 06:51:19,278 - INFO  [main:[email protected]] - minSessionTimeout set to -1
    8. 2012-01-08 06:51:19,279 - INFO  [main:[email protected]] - maxSessionTimeout set to -1
    9. 2012-01-08 06:51:19,281 - INFO  [main:[email protected]] - initLimit set to 5
    10. 2012-01-08 06:51:19,347 - INFO  [Thread-1:[email protected]] - My election bind port: 3888
    11. 2012-01-08 06:51:19,393 - INFO  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:[email protected]] - LOOKING
    12. 2012-01-08 06:51:19,396 - INFO  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:[email protected]] - New election. My id =  1, Proposed zxid = 0
    13. 2012-01-08 06:51:19,400 - INFO  [WorkerReceiver Thread:[email protected]] - Notification: 1 (n.leader), 0 (n.zxid), 1 (n.round), LOOKING (n.state), 1 (n.sid), LOOKING (my state)
    14. 2012-01-08 06:51:19,416 - WARN  [WorkerSender Thread:[email protected]] - Cannot open channel to 2 at election address slave-02/192.168.0.178:3888
    15. java.net.ConnectException: Connection refused
    16. at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
    17. at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)
    18. at sun.nio.ch.SocketAdaptor.connect(SocketAdaptor.java:100)
    19. at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:371)
    20. at org.apache.zookeeper.server.quorum.QuorumCnxManager.toSend(QuorumCnxManager.java:340)
    21. at org.apache.zookeeper.server.quorum.FastLeaderElection$Messenger$WorkerSender.process(FastLeaderElection.java:360)
    22. at org.apache.zookeeper.server.quorum.FastLeaderElection$Messenger$WorkerSender.run(FastLeaderElection.java:333)
    23. at java.lang.Thread.run(Thread.java:662)
    24. 2012-01-08 06:51:19,420 - WARN  [WorkerSender Thread:[email protected]] - Cannot open channel to 3 at election address slave-03/192.168.0.177:3888
    25. java.net.ConnectException: Connection refused
    26. at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
    27. at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)
    28. at sun.nio.ch.SocketAdaptor.connect(SocketAdaptor.java:100)
    29. at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:371)
    30. at org.apache.zookeeper.server.quorum.QuorumCnxManager.toSend(QuorumCnxManager.java:340)
    31. at org.apache.zookeeper.server.quorum.FastLeaderElection$Messenger$WorkerSender.process(FastLeaderElection.java:360)
    32. at org.apache.zookeeper.server.quorum.FastLeaderElection$Messenger$WorkerSender.run(FastLeaderElection.java:333)
    33. at java.lang.Thread.run(Thread.java:662)
    34. 2012-01-08 06:51:19,612 - WARN  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:[email protected]] - Cannot open channel to 2 at election address slave-02/192.168.0.178:3888
    35. java.net.ConnectException: Connection refused
    36. at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
    37. at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)
    38. at sun.nio.ch.SocketAdaptor.connect(SocketAdaptor.java:100)
    39. at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:371)
    40. at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectAll(QuorumCnxManager.java:404)
    41. at org.apache.zookeeper.server.quorum.FastLeaderElection.lookForLeader(FastLeaderElection.java:688)
    42. at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:622)
    43. 2012-01-08 06:51:19,615 - WARN  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:[email protected]] - Cannot open channel to 3 at election address slave-03/192.168.0.177:3888
    44. java.net.ConnectException: Connection refused
    45. at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
    46. at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)
    47. at sun.nio.ch.SocketAdaptor.connect(SocketAdaptor.java:100)
    48. at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:371)
    49. at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectAll(QuorumCnxManager.java:404)
    50. at org.apache.zookeeper.server.quorum.FastLeaderElection.lookForLeader(FastLeaderElection.java:688)
    51. at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:622)
    52. 2012-01-08 06:51:19,616 - INFO  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:[email protected]] - Notification time out: 400
    53. 2012-01-08 06:51:20,019 - WARN  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:[email protected]] - Cannot open channel to 2 at election address slave-02/192.168.0.178:3888
    54. java.net.ConnectException: Connection refused
    55. at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
    56. at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)
    57. at sun.nio.ch.SocketAdaptor.connect(SocketAdaptor.java:100)
    58. at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:371)
    59. at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectAll(QuorumCnxManager.java:404)
    60. at org.apache.zookeeper.server.quorum.FastLeaderElection.lookForLeader(FastLeaderElection.java:688)
    61. at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:622)
    62. 2012-01-08 06:51:20,021 - WARN  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:[email protected]84] - Cannot open channel to 3 at election address slave-03/192.168.0.177:3888
    63. java.net.ConnectException: Connection refused
    64. at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
    65. at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)
    66. at sun.nio.ch.SocketAdaptor.connect(SocketAdaptor.java:100)
    67. at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:371)
    68. at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectAll(QuorumCnxManager.java:404)
    69. at org.apache.zookeeper.server.quorum.FastLeaderElection.lookForLeader(FastLeaderElection.java:688)
    70. at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:622)
    71. 2012-01-08 06:51:20,022 - INFO  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:[email protected]] - Notification time out: 800
    72. 2012-01-08 06:51:20,825 - WARN  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:[email protected]] - Cannot open channel to 2 at election address slave-02/192.168.0.178:3888
    73. java.net.ConnectException: Connection refused
    74. at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
    75. at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)
    76. at sun.nio.ch.SocketAdaptor.connect(SocketAdaptor.java:100)
    77. at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:371)
    78. at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectAll(QuorumCnxManager.java:404)
    79. at org.apache.zookeeper.server.quorum.FastLeaderElection.lookForLeader(FastLeaderElection.java:688)
    80. at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:622)
    81. 2012-01-08 06:51:20,827 - WARN  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:[email protected]] - Cannot open channel to 3 at election address slave-03/192.168.0.177:3888
    82. java.net.ConnectException: Connection refused
    83. at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
    84. at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)
    85. at sun.nio.ch.SocketAdaptor.connect(SocketAdaptor.java:100)
    86. at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:371)
    87. at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectAll(QuorumCnxManager.java:404)
    88. at org.apache.zookeeper.server.quorum.FastLeaderElection.lookForLeader(FastLeaderElection.java:688)
    89. at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:622)
    90. 2012-01-08 06:51:20,828 - INFO  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:[email protected]] - Notification time out: 1600
    91. 2012-01-08 06:51:22,435 - WARN  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:[email protected]] - Cannot open channel to 2 at election address slave-02/192.168.0.178:3888
    92. java.net.ConnectException: Connection refused
    93. at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
    94. at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)
    95. at sun.nio.ch.SocketAdaptor.connect(SocketAdaptor.java:100)
    96. at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:371)
    97. at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectAll(QuorumCnxManager.java:404)
    98. at org.apache.zookeeper.server.quorum.FastLeaderElection.lookForLeader(FastLeaderElection.java:688)
    99. at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:622)
    100. 2012-01-08 06:51:22,439 - WARN  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:[email protected]] - Cannot open channel to 3 at election address slave-03/192.168.0.177:3888
    101. java.net.ConnectException: Connection refused
    102. at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
    103. at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)
    104. at sun.nio.ch.SocketAdaptor.connect(SocketAdaptor.java:100)
    105. at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:371)
    106. at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectAll(QuorumCnxManager.java:404)
    107. at org.apache.zookeeper.server.quorum.FastLeaderElection.lookForLeader(FastLeaderElection.java:688)
    108. at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:622)
    109. 2012-01-08 06:51:22,441 - INFO  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:[email protected]] - Notification time out: 3200
    110. 2012-01-08 06:51:22,945 - INFO  [WorkerReceiver Thread:[email protected]] - Notification: 2 (n.leader), 0 (n.zxid), 1 (n.round), LOOKING (n.state), 2 (n.sid), LOOKING (my state)
    111. 2012-01-08 06:51:22,946 - INFO  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:[email protected]] - Updating proposal
    112. 2012-01-08 06:51:22,949 - WARN  [WorkerSender Thread:[email protected]] - Cannot open channel to 3 at election address slave-03/192.168.0.177:3888
    113. java.net.ConnectException: Connection refused
    114. at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
    115. at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)
    116. at sun.nio.ch.SocketAdaptor.connect(SocketAdaptor.java:100)
    117. at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:371)
    118. at org.apache.zookeeper.server.quorum.QuorumCnxManager.toSend(QuorumCnxManager.java:340)
    119. at org.apache.zookeeper.server.quorum.FastLeaderElection$Messenger$WorkerSender.process(FastLeaderElection.java:360)
    120. at org.apache.zookeeper.server.quorum.FastLeaderElection$Messenger$WorkerSender.run(FastLeaderElection.java:333)
    121. at java.lang.Thread.run(Thread.java:662)
    122. 2012-01-08 06:51:22,951 - INFO  [WorkerReceiver Thread:[email protected]] - Notification: 2 (n.leader), 0 (n.zxid), 1 (n.round), LOOKING (n.state), 1 (n.sid), LOOKING (my state)
    123. 2012-01-08 06:51:23,156 - INFO  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:[email protected]] - FOLLOWING
    124. 2012-01-08 06:51:23,170 - INFO  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:[email protected]] - TCP NoDelay set to: true
    125. 2012-01-08 06:51:23,206 - INFO  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:[email protected]] - Server environment:zookeeper.version=3.3.3-1203054, built on 11/17/2011 05:47 GMT
    126. 2012-01-08 06:51:23,207 - INFO  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:[email protected]] - Server environment:host.name=slave-01
    127. 2012-01-08 06:51:23,207 - INFO  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:[email protected]] - Server environment:java.version=1.6.0_30
    128. 2012-01-08 06:51:23,208 - INFO  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:[email protected]] - Server environment:java.vendor=Sun Microsystems Inc.
    129. 2012-01-08 06:51:23,208 - INFO  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:[email protected]] - Server environment:java.home=/home/hadoop/installation/jdk1.6.0_30/jre
    130. 2012-01-08 06:51:23,209 - INFO  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:[email protected]] - Server environment:java.class.path=/home/hadoop/installation/zookeeper-3.3.4/bin/../build/classes:/home/hadoop/installation/zookeeper-3.3.4/bin/../build/lib/*.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../zookeeper-3.3.4.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../lib/log4j-1.2.15.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../lib/jline-0.9.94.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../lib/commons-lang-2.4.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../lib/commons-collections-3.2.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../lib/commons-cli-1.1.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../lib/apache-rat-tasks-0.6.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../lib/apache-rat-core-0.6.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../src/java/lib/*.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../conf:/home/hadoop/installation/jdk1.6.0_30/lib/*.jar:/home/hadoop/installation/jdk1.6.0_30/jre/lib/*.jar
    131. 2012-01-08 06:51:23,210 - INFO  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:[email protected]] - Server environment:java.library.path=/home/hadoop/installation/jdk1.6.0_30/jre/lib/i386/client:/home/hadoop/installation/jdk1.6.0_30/jre/lib/i386:/home/hadoop/installation/jdk1.6.0_30/jre/../lib/i386:/usr/java/packages/lib/i386:/lib:/usr/lib
    132. 2012-01-08 06:51:23,210 - INFO  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:[email protected]] - Server environment:java.io.tmpdir=/tmp
    133. 2012-01-08 06:51:23,212 - INFO  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:[email protected]] - Server environment:java.compiler=<NA>
    134. 2012-01-08 06:51:23,212 - INFO  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:[email protected]] - Server environment:os.name=Linux
    135. 2012-01-08 06:51:23,212 - INFO  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:[email protected]] - Server environment:os.arch=i386
    136. 2012-01-08 06:51:23,213 - INFO  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:[email protected]] - Server environment:os.version=3.0.0-14-generic
    137. 2012-01-08 06:51:23,213 - INFO  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:[email protected]] - Server environment:user.name=hadoop
    138. 2012-01-08 06:51:23,214 - INFO  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:[email protected]] - Server environment:user.home=/home/hadoop
    139. 2012-01-08 06:51:23,214 - INFO  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:[email protected]] - Server environment:user.dir=/home/hadoop/installation/zookeeper-3.3.4
    140. 2012-01-08 06:51:23,223 - INFO  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:[email protected]] - Created server with tickTime 2000 minSessionTimeout 4000 maxSessionTimeout 40000 datadir /home/hadoop/storage/zookeeper/version-2 snapdir /home/hadoop/storage/zookeeper/version-2
    141. 2012-01-08 06:51:23,339 - INFO  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:[email protected]] - Getting a snapshot from leader
    142. 2012-01-08 06:51:23,358 - INFO  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:[email protected]] - Setting leader epoch 1
    143. 2012-01-08 06:51:23,358 - INFO  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:[email protected]] - Snapshotting: 0
    144. 2012-01-08 06:51:25,511 - INFO  [WorkerReceiver Thread:[email protected]] - Notification: 3 (n.leader), 0 (n.zxid), 1 (n.round), LOOKING (n.state), 3 (n.sid), FOLLOWING (my state)
    145. 2012-01-08 06:51:42,584 - INFO  [WorkerReceiver Thread:[email protected]] - Notification: 3 (n.leader), 0 (n.zxid), 2 (n.round), LOOKING (n.state), 3 (n.sid), FOLLOWING (my state)

我 启动的顺序是slave-01>slave-02>slave-03，由于ZooKeeper集群启动的时候，每个结点都试图去连接集群中的 其它结点，先启动的肯定连不上后面还没启动的，所以上面日志前面部分的异常是可以忽略的。通过后面部分可以看到，集群在选出一个Leader后，最后稳定 了。其他结点可能也出现类似问题，属于正常。

第六步：安装验证

可以通过ZooKeeper的脚本来查看启动状态，包括集群中各个结点的角色（或是Leader，或是Follower），如下所示，是在ZooKeeper集群中的每个结点上查询的结果：

　　　

　　　　

时间： 2024-10-03 15:42:49

Zookeeper + Hadoop + Hbase部署备忘的相关文章

在Centos7下搭建大数据环境，即Zookeeper+Hadoop+HBase

1. 所需软件下载链接(建议直接复制链接到迅雷下载更快): ①hadoop-2.7.6.tar.gz: wget http://mirrors.tuna.tsinghua.edu.cn/apache/hadoop/common/hadoop-2.7.6/hadoop-2.7.6.tar.gz ②zookeeper-3.4.12.tar.gz: wget https://mirrors.tuna.tsinghua.edu.cn/apache/zookeeper/stable/zookeeper-3

hbase命令备忘

http://www.cnblogs.com/linjiqin/archive/2013/03/08/2949339.html HBase 为用户提供了一个非常方便的使用方式, 我们称之为“HBase Shell”.HBase Shell 提供了大多数的 HBase 命令, 通过 HBase Shell 用户可以方便地创建.删除及修改表, 还可以向表中添加数据.列出表中的相关信息等.备注:写错 HBase Shell 命令时用键盘上的“Delete”进行删除,“Backspace”不起作用.在启

[推荐]Hadoop+HBase+Zookeeper集群的配置

Hadoop+HBase+Zookeeper集群的配置  http://wenku.baidu.com/view/991258e881c758f5f61f67cc.html?re=view HDFS+MapReduce+Hive+HBase十分钟快速入门   http://wenku.baidu.com/view/7db2fcd276eeaeaad1f33055.html Hadoop+Zookeeper+HBase部署指南  http://wenku.baidu.com/view/02e4ad

Hadoop集群之 ZooKeeper和Hbase环境搭建

前面讲解了Hadoop完全分布式的搭建,本章主要讲解一下在Hadoop完全分布式已经搭建成功的情况下搭建ZooKeeper和Hbase环境 所有软件下载百度云 密码:uup8 讲在开头:对于笔者的完全分布式环境请见该文:Hadoop完全分布式安装 1.ZooKeeper和Hbase的简答介绍: ZooKeeper是一个分布式的,开放源码的分布式应用程序协调服务,是Google的Chubby一个开源的实现,是Hadoop和Hbase的重要组件.它是一个为分布式应用提供一致性服务的软件,提供的功能包

学习搭建Hadoop+HBase+ZooKeeper分布式集群环境

一.环境配置 由于集群至少需要三台服务器,我就拿上次做的MongoDB Master, Slave, Arbiter环境来做Hadoop集群.服务器还是ibmcloud 免费提供的.其中Arbiter在这里做的也是slave的角色. Hostname IP  Server Type Master 192.168.0.28 Centos6.2 Slave 192.168.0.29 Ubuntu14.04 Arbiter 192.168.0.30 Ubuntu14.04 配置三台机器的Master

Hadoop,HBase,Zookeeper源码编译并导入eclipse

基本理念:尽可能的参考官方英文文档 Hadoop:  http://wiki.apache.org/hadoop/FrontPage HBase:  http://hbase.apache.org/book.html Zookeeper:  https://cwiki.apache.org/confluence/display/ZOOKEEPER/Index 环境介绍 Ubuntu 14.04LTS, 32位 接下来则按照Hadoop,HBase,Zookeeper顺序来进行源码编译,建立文件夹

hadoop+hbase+zookeeper+spark+phoenix相关实施报错处理

排错日志: 解决办法:可能是修改 机器名导致的,修改hosts,写入hostname和IP,然后,try it agin! 解决办法: 当引入hadoop-common-2.2.0.jar包进行二次开发,比如读写HDFS文件时,初次运行报错. java.io.IOException: No FileSystem for scheme: hdfs at org.apache.hadoop.fs.FileSystem.getFileSystemClass(FileSystem.java:2421)

备忘zookeeper(单机+伪集群+集群)

#下载: #单机模式 解压到合适目录. 进入zookeeper目录下的conf子目录, 复制zoo_sample.cfg-->zoo.cfg(如果没有data和logs就新建):tickTime=2000 dataDir=/Users/apple/zookeeper/data dataLogDir=/Users/apple/zookeeper/logs clientPort=2180 参数说明:tickTime: zookeeper中使用的基本时间单位, 毫秒值.dataDir: 数据目录. 可

【HBase】zookeeper在HBase中的应用

转自:http://support.huawei.com/ecommunity/bbs/10242721.html Zookeeper在HBase中的应用 HBase部署相对是一个较大的动作,其依赖于zookeeper cluster,hadoop HDFS. Zookeeper作用在于: 1.hbase regionserver 向zookeeper注册,提供hbase regionserver状态信息(是否在线). 2.hmaster启动时候会将hbase系统表-ROOT- 加载到 zook