zookeeper 集群安装（单点与分布式成功安装）摘录

http://www.blogjava.net/hello-yun/archive/2012/05/03/377250.html

ZooKeeper是一个分布式开源框架，提供了协调分布式应用的基本服务，它向外部应用暴露一组通用服务——分布式同步（Distributed Synchronization）、命名服务（Naming Service）、集群维护（Group Maintenance）等，简化分布式应用协调及其管理的难度，提供高性能的分布式服务。ZooKeeper本身可以以Standalone模式安装运行，不过它的长处在于通过分布式ZooKeeper集群（一个Leader，多个Follower），基于一定的策略来保证ZooKeeper集群的稳定性和可用性，从而实现分布式应用的可靠性。

有关ZooKeeper的介绍，网上很多，也可以参考文章后面，我整理的一些相关链接。

下面，我们简单说明一下ZooKeeper的配置。

ZooKeeper Standalone模式

从Apache网站上（zookeeper.apache.org）下载ZooKeeper软件包，我选择了3.3.4版本的（zookeeper-3.3.4.tar.gz），在一台Linux机器上安装非常容易，只需要解压缩后，简单配置一下即可以启动ZooKeeper服务器进程。

将zookeeper-3.3.4/conf目录下面的 zoo_sample.cfg修改为zoo.cfg，配置文件内容如下所示：

[plain] view plaincopy

  1. tickTime=2000
  2. dataDir=/home/hadoop/storage/zookeeper
  3. clientPort=2181
  4. initLimit=5
  5. syncLimit=2

上面各个配置参数的含义也非常简单，引用如下所示：

[plain] view plaincopy

  1. tickTime —— the basic time unit in milliseconds used by ZooKeeper. It is used to do heartbeats and the minimum session timeout will be twice the tickTime.
  2. dataDir —— the location to store the in-memory database snapshots and, unless specified otherwise, the transaction log of updates to the database.
  3. clientPort —— the port to listen for client connections

下面启动ZooKeeper服务器进程：

[plain] view plaincopy

  1. cd zookeeper-3.3.4/
  2. bin/zkServer.sh start

通过jps命令可以查看ZooKeeper服务器进程，名称为QuorumPeerMain。

在客户端连接ZooKeeper服务器，执行如下命令：

[plain] view plaincopy

  1. bin/zkCli.sh -server dynamic:2181

上面dynamic是我的主机名，如果在本机执行，则执行如下命令即可：

[plain] view plaincopy

  1. bin/zkCli.sh

客户端连接信息如下所示：接着，可以使用help查看Zookeeper客户端可以使用的基本操作命令。

ZooKeeper Distributed模式

ZooKeeper分布式模式安装（ZooKeeper集群）也比较容易，这里说明一下基本要点。

首先要明确的是，ZooKeeper集群是一个独立的分布式协调服务集群，“独立”的含义就是说，如果想使用ZooKeeper实现分布式应用的协调与管理，简化协调与管理，任何分布式应用都可以使用，这就要归功于Zookeeper的数据模型（Data Model）和层次命名空间（Hierarchical Namespace）结构，详细可以参考http://zookeeper.apache.org/doc/trunk/zookeeperOver.html。在设计你的分布式应用协调服务时，首要的就是考虑如何组织层次命名空间。

下面说明分布式模式的安装配置，过程如下所示：

第一步：主机名称到IP地址映射配置

ZooKeeper集群中具有两个关键的角色：Leader和Follower。集群中所有的结点作为一个整体对分布式应用提供服务，集群中每个结点之间都互相连接，所以，在配置的ZooKeeper集群的时候，每一个结点的host到IP地址的映射都要配置上集群中其它结点的映射信息。

例如，我的ZooKeeper集群中每个结点的配置，以slave-01为例，/etc/hosts内容如下所示：

[plain] view plaincopy

  1. 192.168.0.179   slave-01
  2. 192.168.0.178   slave-02
  3. 192.168.0.177   slave-03

ZooKeeper采用一种称为Leader election的选举算法。在整个集群运行过程中，只有一个Leader，其他的都是Follower，如果ZooKeeper集群在运行过程中Leader出了问题，系统会采用该算法重新选出一个Leader。因此，各个结点之间要能够保证互相连接，必须配置上述映射。

ZooKeeper集群启动的时候，会首先选出一个Leader，在Leader election过程中，某一个满足选举算的结点就能成为Leader。整个集群的架构可以参考http://zookeeper.apache.org/doc/trunk/zookeeperOver.html#sc_designGoals。

第二步：修改ZooKeeper配置文件

在其中一台机器（slave-01）上，解压缩zookeeper-3.3.4.tar.gz，修改配置文件conf/zoo.cfg，内容如下所示：

[plain] view plaincopy

  1. tickTime=2000
  2. dataDir=/home/hadoop/storage/zookeeper
  3. clientPort=2181
  4. initLimit=5
  5. syncLimit=2
  6. server.1=slave-01:2888:3888
  7. server.2=slave-02:2888:3888
  8. server.3=slave-03:2888:3888

上述配置内容说明，可以参考http://zookeeper.apache.org/doc/trunk/zookeeperStarted.html#sc_RunningReplicatedZooKeeper。

第三步：远程复制分发安装文件

上面已经在一台机器slave-01上配置完成ZooKeeper，现在可以将该配置好的安装文件远程拷贝到集群中的各个结点对应的目录下：

[plain] view plaincopy

  1. cd /home/hadoop/installation/
  2. scp -r zookeeper-3.3.4/ [email protected]:/home/hadoop/installation/
  3. scp -r zookeeper-3.3.4/ [email protected]:/home/hadoop/installation/

第四步：设置myid

在我们配置的dataDir指定的目录下面，创建一个myid文件，里面内容为一个数字，用来标识当前主机，conf/zoo.cfg文件中配置的server.X中X为什么数字，则myid文件中就输入这个数字，例如：

[plain] view plaincopy

  1. [email protected]:~/installation/zookeeper-3.3.4$ echo "1" > /home/hadoop/storage/zookeeper/myid
  2. [email protected]:~/installation/zookeeper-3.3.4$ echo "2" > /home/hadoop/storage/zookeeper/myid
  3. [email protected]:~/installation/zookeeper-3.3.4$ echo "3" > /home/hadoop/storage/zookeeper/myid

按照上述进行配置即可。

第五步：启动ZooKeeper集群

在ZooKeeper集群的每个结点上，执行启动ZooKeeper服务的脚本，如下所示：

[plain] view plaincopy

  1. [email protected]:~/installation/zookeeper-3.3.4$ bin/zkServer.sh start
  2. [email protected]:~/installation/zookeeper-3.3.4$ bin/zkServer.sh start
  3. [email protected]:~/installation/zookeeper-3.3.4$ bin/zkServer.sh start

以结点slave-01为例，日志如下所示：

[plain] view plaincopy

  1. [email protected]:~/installation/zookeeper-3.3.4$ tail -500f zookeeper.out
  2. 2012-01-08 06:51:19,117 - INFO  [main:[email protected]] - Reading configuration from: /home/hadoop/installation/zookeeper-3.3.4/bin/../conf/zoo.cfg
  3. 2012-01-08 06:51:19,133 - INFO  [main:[email protected]] - Defaulting to majority quorums
  4. 2012-01-08 06:51:19,167 - INFO  [main:[email protected]] - Starting quorum peer
  5. 2012-01-08 06:51:19,227 - INFO  [main:[email protected]] - binding to port 0.0.0.0/0.0.0.0:2181
  6. 2012-01-08 06:51:19,277 - INFO  [main:[email protected]] - tickTime set to 2000
  7. 2012-01-08 06:51:19,278 - INFO  [main:[email protected]] - minSessionTimeout set to -1
  8. 2012-01-08 06:51:19,279 - INFO  [main:[email protected]] - maxSessionTimeout set to -1
  9. 2012-01-08 06:51:19,281 - INFO  [main:[email protected]] - initLimit set to 5
  10. 2012-01-08 06:51:19,347 - INFO  [Thread-1:[email protected]] - My election bind port: 3888
  11. 2012-01-08 06:51:19,393 - INFO  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:[email protected]] - LOOKING
  12. 2012-01-08 06:51:19,396 - INFO  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:[email protected]] - New election. My id =  1, Proposed zxid = 0
  13. 2012-01-08 06:51:19,400 - INFO  [WorkerReceiver Thread:[email protected]] - Notification: 1 (n.leader), 0 (n.zxid), 1 (n.round), LOOKING (n.state), 1 (n.sid), LOOKING (my state)
  14. 2012-01-08 06:51:19,416 - WARN  [WorkerSender Thread:[email protected]] - Cannot open channel to 2 at election address slave-02/192.168.0.178:3888
  15. java.net.ConnectException: Connection refused
  16. at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
  17. at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)
  18. at sun.nio.ch.SocketAdaptor.connect(SocketAdaptor.java:100)
  19. at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:371)
  20. at org.apache.zookeeper.server.quorum.QuorumCnxManager.toSend(QuorumCnxManager.java:340)
  21. at org.apache.zookeeper.server.quorum.FastLeaderElection$Messenger$WorkerSender.process(FastLeaderElection.java:360)
  22. at org.apache.zookeeper.server.quorum.FastLeaderElection$Messenger$WorkerSender.run(FastLeaderElection.java:333)
  23. at java.lang.Thread.run(Thread.java:662)
  24. 2012-01-08 06:51:19,420 - WARN  [WorkerSender Thread:[email protected]] - Cannot open channel to 3 at election address slave-03/192.168.0.177:3888
  25. java.net.ConnectException: Connection refused
  26. at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
  27. at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)
  28. at sun.nio.ch.SocketAdaptor.connect(SocketAdaptor.java:100)
  29. at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:371)
  30. at org.apache.zookeeper.server.quorum.QuorumCnxManager.toSend(QuorumCnxManager.java:340)
  31. at org.apache.zookeeper.server.quorum.FastLeaderElection$Messenger$WorkerSender.process(FastLeaderElection.java:360)
  32. at org.apache.zookeeper.server.quorum.FastLeaderElection$Messenger$WorkerSender.run(FastLeaderElection.java:333)
  33. at java.lang.Thread.run(Thread.java:662)
  34. 2012-01-08 06:51:19,612 - WARN  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:[email protected]] - Cannot open channel to 2 at election address slave-02/192.168.0.178:3888
  35. java.net.ConnectException: Connection refused
  36. at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
  37. at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)
  38. at sun.nio.ch.SocketAdaptor.connect(SocketAdaptor.java:100)
  39. at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:371)
  40. at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectAll(QuorumCnxManager.java:404)
  41. at org.apache.zookeeper.server.quorum.FastLeaderElection.lookForLeader(FastLeaderElection.java:688)
  42. at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:622)
  43. 2012-01-08 06:51:19,615 - WARN  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:[email protected]] - Cannot open channel to 3 at election address slave-03/192.168.0.177:3888
  44. java.net.ConnectException: Connection refused
  45. at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
  46. at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)
  47. at sun.nio.ch.SocketAdaptor.connect(SocketAdaptor.java:100)
  48. at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:371)
  49. at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectAll(QuorumCnxManager.java:404)
  50. at org.apache.zookeeper.server.quorum.FastLeaderElection.lookForLeader(FastLeaderElection.java:688)
  51. at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:622)
  52. 2012-01-08 06:51:19,616 - INFO  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:[email protected]] - Notification time out: 400
  53. 2012-01-08 06:51:20,019 - WARN  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:[email protected]] - Cannot open channel to 2 at election address slave-02/192.168.0.178:3888
  54. java.net.ConnectException: Connection refused
  55. at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
  56. at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)
  57. at sun.nio.ch.SocketAdaptor.connect(SocketAdaptor.java:100)
  58. at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:371)
  59. at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectAll(QuorumCnxManager.java:404)
  60. at org.apache.zookeeper.server.quorum.FastLeaderElection.lookForLeader(FastLeaderElection.java:688)
  61. at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:622)
  62. 2012-01-08 06:51:20,021 - WARN  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:[email protected]] - Cannot open channel to 3 at election address slave-03/192.168.0.177:3888
  63. java.net.ConnectException: Connection refused
  64. at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
  65. at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)
  66. at sun.nio.ch.SocketAdaptor.connect(SocketAdaptor.java:100)
  67. at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:371)
  68. at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectAll(QuorumCnxManager.java:404)
  69. at org.apache.zookeeper.server.quorum.FastLeaderElection.lookForLeader(FastLeaderElection.java:688)
  70. at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:622)
  71. 2012-01-08 06:51:20,022 - INFO  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:[email protected]] - Notification time out: 800
  72. 2012-01-08 06:51:20,825 - WARN  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:[email protected]] - Cannot open channel to 2 at election address slave-02/192.168.0.178:3888
  73. java.net.ConnectException: Connection refused
  74. at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
  75. at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)
  76. at sun.nio.ch.SocketAdaptor.connect(SocketAdaptor.java:100)
  77. at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:371)
  78. at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectAll(QuorumCnxManager.java:404)
  79. at org.apache.zookeeper.server.quorum.FastLeaderElection.lookForLeader(FastLeaderElection.java:688)
  80. at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:622)
  81. 2012-01-08 06:51:20,827 - WARN  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:[email protected]] - Cannot open channel to 3 at election address slave-03/192.168.0.177:3888
  82. java.net.ConnectException: Connection refused
  83. at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
  84. at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)
  85. at sun.nio.ch.SocketAdaptor.connect(SocketAdaptor.java:100)
  86. at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:371)
  87. at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectAll(QuorumCnxManager.java:404)
  88. at org.apache.zookeeper.server.quorum.FastLeaderElection.lookForLeader(FastLeaderElection.java:688)
  89. at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:622)
  90. 2012-01-08 06:51:20,828 - INFO  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:[email protected]] - Notification time out: 1600
  91. 2012-01-08 06:51:22,435 - WARN  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:[email protected]] - Cannot open channel to 2 at election address slave-02/192.168.0.178:3888
  92. java.net.ConnectException: Connection refused
  93. at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
  94. at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)
  95. at sun.nio.ch.SocketAdaptor.connect(SocketAdaptor.java:100)
  96. at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:371)
  97. at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectAll(QuorumCnxManager.java:404)
  98. at org.apache.zookeeper.server.quorum.FastLeaderElection.lookForLeader(FastLeaderElection.java:688)
  99. at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:622)
  100. 2012-01-08 06:51:22,439 - WARN  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:[email protected]] - Cannot open channel to 3 at election address slave-03/192.168.0.177:3888
  101. java.net.ConnectException: Connection refused
  102. at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
  103. at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)
  104. at sun.nio.ch.SocketAdaptor.connect(SocketAdaptor.java:100)
  105. at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:371)
  106. at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectAll(QuorumCnxManager.java:404)
  107. at org.apache.zookeeper.server.quorum.FastLeaderElection.lookForLeader(FastLeaderElection.java:688)
  108. at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:622)
  109. 2012-01-08 06:51:22,441 - INFO  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:[email protected]] - Notification time out: 3200
  110. 2012-01-08 06:51:22,945 - INFO  [WorkerReceiver Thread:[email protected]] - Notification: 2 (n.leader), 0 (n.zxid), 1 (n.round), LOOKING (n.state), 2 (n.sid), LOOKING (my state)
  111. 2012-01-08 06:51:22,946 - INFO  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:[email protected]] - Updating proposal
  112. 2012-01-08 06:51:22,949 - WARN  [WorkerSender Thread:[email protected]] - Cannot open channel to 3 at election address slave-03/192.168.0.177:3888
  113. java.net.ConnectException: Connection refused
  114. at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
  115. at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)
  116. at sun.nio.ch.SocketAdaptor.connect(SocketAdaptor.java:100)
  117. at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:371)
  118. at org.apache.zookeeper.server.quorum.QuorumCnxManager.toSend(QuorumCnxManager.java:340)
  119. at org.apache.zookeeper.server.quorum.FastLeaderElection$Messenger$WorkerSender.process(FastLeaderElection.java:360)
  120. at org.apache.zookeeper.server.quorum.FastLeaderElection$Messenger$WorkerSender.run(FastLeaderElection.java:333)
  121. at java.lang.Thread.run(Thread.java:662)
  122. 2012-01-08 06:51:22,951 - INFO  [WorkerReceiver Thread:[email protected]] - Notification: 2 (n.leader), 0 (n.zxid), 1 (n.round), LOOKING (n.state), 1 (n.sid), LOOKING (my state)
  123. 2012-01-08 06:51:23,156 - INFO  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:[email protected]] - FOLLOWING
  124. 2012-01-08 06:51:23,170 - INFO  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:[email protected]] - TCP NoDelay set to: true
  125. 2012-01-08 06:51:23,206 - INFO  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:[email protected]] - Server environment:zookeeper.version=3.3.3-1203054, built on 11/17/2011 05:47 GMT
  126. 2012-01-08 06:51:23,207 - INFO  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:[email protected]] - Server environment:host.name=slave-01
  127. 2012-01-08 06:51:23,207 - INFO  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:[email protected]] - Server environment:java.version=1.6.0_30
  128. 2012-01-08 06:51:23,208 - INFO  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:[email protected]] - Server environment:java.vendor=Sun Microsystems Inc.
  129. 2012-01-08 06:51:23,208 - INFO  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:[email protected]] - Server environment:java.home=/home/hadoop/installation/jdk1.6.0_30/jre
  130. 2012-01-08 06:51:23,209 - INFO  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:[email protected]] - Server environment:java.class.path=/home/hadoop/installation/zookeeper-3.3.4/bin/../build/classes:/home/hadoop/installation/zookeeper-3.3.4/bin/../build/lib/*.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../zookeeper-3.3.4.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../lib/log4j-1.2.15.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../lib/jline-0.9.94.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../lib/commons-lang-2.4.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../lib/commons-collections-3.2.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../lib/commons-cli-1.1.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../lib/apache-rat-tasks-0.6.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../lib/apache-rat-core-0.6.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../src/java/lib/*.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../conf:/home/hadoop/installation/jdk1.6.0_30/lib/*.jar:/home/hadoop/installation/jdk1.6.0_30/jre/lib/*.jar
  131. 2012-01-08 06:51:23,210 - INFO  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:[email protected]] - Server environment:java.library.path=/home/hadoop/installation/jdk1.6.0_30/jre/lib/i386/client:/home/hadoop/installation/jdk1.6.0_30/jre/lib/i386:/home/hadoop/installation/jdk1.6.0_30/jre/../lib/i386:/usr/java/packages/lib/i386:/lib:/usr/lib
  132. 2012-01-08 06:51:23,210 - INFO  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:[email protected]] - Server environment:java.io.tmpdir=/tmp
  133. 2012-01-08 06:51:23,212 - INFO  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:[email protected]] - Server environment:java.compiler=<NA>
  134. 2012-01-08 06:51:23,212 - INFO  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:[email protected]] - Server environment:os.name=Linux
  135. 2012-01-08 06:51:23,212 - INFO  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:[email protected]] - Server environment:os.arch=i386
  136. 2012-01-08 06:51:23,213 - INFO  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:[email protected]] - Server environment:os.version=3.0.0-14-generic
  137. 2012-01-08 06:51:23,213 - INFO  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:[email protected]] - Server environment:user.name=hadoop
  138. 2012-01-08 06:51:23,214 - INFO  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:[email protected]] - Server environment:user.home=/home/hadoop
  139. 2012-01-08 06:51:23,214 - INFO  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:[email protected]] - Server environment:user.dir=/home/hadoop/installation/zookeeper-3.3.4
  140. 2012-01-08 06:51:23,223 - INFO  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:[email protected]] - Created server with tickTime 2000 minSessionTimeout 4000 maxSessionTimeout 40000 datadir /home/hadoop/storage/zookeeper/version-2 snapdir /home/hadoop/storage/zookeeper/version-2
  141. 2012-01-08 06:51:23,339 - INFO  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:[email protected]] - Getting a snapshot from leader
  142. 2012-01-08 06:51:23,358 - INFO  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:[email protected]] - Setting leader epoch 1
  143. 2012-01-08 06:51:23,358 - INFO  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:[email protected]] - Snapshotting: 0
  144. 2012-01-08 06:51:25,511 - INFO  [WorkerReceiver Thread:[email protected]] - Notification: 3 (n.leader), 0 (n.zxid), 1 (n.round), LOOKING (n.state), 3 (n.sid), FOLLOWING (my state)
  145. 2012-01-08 06:51:42,584 - INFO  [WorkerReceiver Thread:[email protected]] - Notification: 3 (n.leader), 0 (n.zxid), 2 (n.round), LOOKING (n.state), 3 (n.sid), FOLLOWING (my state)

我启动的顺序是slave-01>slave-02>slave-03，由于ZooKeeper集群启动的时候，每个结点都试图去连接集群中的其它结点，先启动的肯定连不上后面还没启动的，所以上面日志前面部分的异常是可以忽略的。通过后面部分可以看到，集群在选出一个Leader后，最后稳定了。

其他结点可能也出现类似问题，属于正常。

第六步：安装验证

可以通过ZooKeeper的脚本来查看启动状态，包括集群中各个结点的角色（或是Leader，或是Follower），如下所示，是在ZooKeeper集群中的每个结点上查询的结果：

[plain] view plaincopy

  1. [email protected]:~/installation/zookeeper-3.3.4$ bin/zkServer.sh status
  2. JMX enabled by default
  3. Using config: /home/hadoop/installation/zookeeper-3.3.4/bin/../conf/zoo.cfg
  4. Mode: follower
  5. [email protected]:~/installation/zookeeper-3.3.4$  bin/zkServer.sh status
  6. JMX enabled by default
  7. Using config: /home/hadoop/installation/zookeeper-3.3.4/bin/../conf/zoo.cfg
  8. Mode: leader
  9. [email protected]:~/installation/zookeeper-3.3.4$  bin/zkServer.sh status
  10. JMX enabled by default
  11. Using config: /home/hadoop/installation/zookeeper-3.3.4/bin/../conf/zoo.cfg
  12. Mode: follower

通过上面状态查询结果可见，slave-02是集群的Leader，其余的两个结点是Follower。

另外，可以通过客户端脚本，连接到ZooKeeper集群上。对于客户端来说，ZooKeeper是一个整体（ensemble），连接到ZooKeeper集群实际上感觉在独享整个集群的服务，所以，你可以在任何一个结点上建立到服务集群的连接，例如：

[plain] view plaincopy

  1. [email protected]:~/installation/zookeeper-3.3.4$ bin/zkCli.sh -server slave-01:2181
  2. Connecting to slave-01:2181
  3. 2012-01-08 07:14:21,068 - INFO  [main:[email protected]] - Client environment:zookeeper.version=3.3.3-1203054, built on 11/17/2011 05:47 GMT
  4. 2012-01-08 07:14:21,080 - INFO  [main:[email protected]] - Client environment:host.name=slave-03
  5. 2012-01-08 07:14:21,085 - INFO  [main:[email protected]] - Client environment:java.version=1.6.0_30
  6. 2012-01-08 07:14:21,089 - INFO  [main:[email protected]] - Client environment:java.vendor=Sun Microsystems Inc.
  7. 2012-01-08 07:14:21,095 - INFO  [main:[email protected]] - Client environment:java.home=/home/hadoop/installation/jdk1.6.0_30/jre
  8. 2012-01-08 07:14:21,104 - INFO  [main:[email protected]] - Client environment:java.class.path=/home/hadoop/installation/zookeeper-3.3.4/bin/../build/classes:/home/hadoop/installation/zookeeper-3.3.4/bin/../build/lib/*.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../zookeeper-3.3.4.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../lib/log4j-1.2.15.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../lib/jline-0.9.94.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../lib/commons-lang-2.4.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../lib/commons-collections-3.2.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../lib/commons-cli-1.1.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../lib/apache-rat-tasks-0.6.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../lib/apache-rat-core-0.6.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../src/java/lib/*.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../conf:/home/hadoop/installation/jdk1.6.0_30/lib/*.jar:/home/hadoop/installation/jdk1.6.0_30/jre/lib/*.jar
  9. 2012-01-08 07:14:21,111 - INFO  [main:[email protected]] - Client environment:java.library.path=/home/hadoop/installation/jdk1.6.0_30/jre/lib/i386/client:/home/hadoop/installation/jdk1.6.0_30/jre/lib/i386:/home/hadoop/installation/jdk1.6.0_30/jre/../lib/i386:/usr/java/packages/lib/i386:/lib:/usr/lib
  10. 2012-01-08 07:14:21,116 - INFO  [main:[email protected]] - Client environment:java.io.tmpdir=/tmp
  11. 2012-01-08 07:14:21,124 - INFO  [main:[email protected]] - Client environment:java.compiler=<NA>
  12. 2012-01-08 07:14:21,169 - INFO  [main:[email protected]] - Client environment:os.name=Linux
  13. 2012-01-08 07:14:21,175 - INFO  [main:[email protected]] - Client environment:os.arch=i386
  14. 2012-01-08 07:14:21,177 - INFO  [main:[email protected]] - Client environment:os.version=3.0.0-14-generic
  15. 2012-01-08 07:14:21,185 - INFO  [main:[email protected]] - Client environment:user.name=hadoop
  16. 2012-01-08 07:14:21,188 - INFO  [main:[email protected]] - Client environment:user.home=/home/hadoop
  17. 2012-01-08 07:14:21,190 - INFO  [main:[email protected]] - Client environment:user.dir=/home/hadoop/installation/zookeeper-3.3.4
  18. 2012-01-08 07:14:21,197 - INFO  [main:[email protected]] - Initiating client connection, connectString=slave-01:2181 sessionTimeout=30000 [email protected]
  19. 2012-01-08 07:14:21,305 - INFO  [main-SendThread():[email protected]] - Opening socket connection to server slave-01/192.168.0.179:2181
  20. Welcome to ZooKeeper!
  21. 2012-01-08 07:14:21,376 - INFO  [main-SendThread(slave-01:2181):[email protected]] - Socket connection established to slave-01/192.168.0.179:2181, initiating session
  22. JLine support is enabled
  23. [zk: slave-01:2181(CONNECTING) 0] 2012-01-08 07:14:21,872 - INFO  [main-SendThread(slave-01:2181):[email protected]] - Session establishment complete on server slave-01/192.168.0.179:2181, sessionid = 0x134bdcd6b730000, negotiated timeout = 30000
  24. WATCHER::
  25. WatchedEvent state:SyncConnected type:None path:null
  26. [zk: slave-01:2181(CONNECTED) 0] ls /
  27. [zookeeper]

当前根路径为/zookeeper。

总结说明

主机名与IP地址映射配置问题

启动ZooKeeper集群时，如果ZooKeeper集群中slave-01结点的日志出现如下错误：

[plain] view plaincopy

  1. java.net.SocketTimeoutException
  2. at sun.nio.ch.SocketAdaptor.connect(SocketAdaptor.java:109)
  3. at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:371)
  4. at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectAll(QuorumCnxManager.java:404)
  5. at org.apache.zookeeper.server.quorum.FastLeaderElection.lookForLeader(FastLeaderElection.java:688)
  6. at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:622)
  7. 2012-01-08 06:37:46,026 - INFO  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:[email protected]] - Notification time out: 6400
  8. 2012-01-08 06:37:57,431 - WARN  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:[email protected]] - Cannot open channel to 2 at election address slave-02/202.106.199.35:3888
  9. java.net.SocketTimeoutException
  10. at sun.nio.ch.SocketAdaptor.connect(SocketAdaptor.java:109)
  11. at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:371)
  12. at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectAll(QuorumCnxManager.java:404)
  13. at org.apache.zookeeper.server.quorum.FastLeaderElection.lookForLeader(FastLeaderElection.java:688)
  14. at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:622)
  15. 2012-01-08 06:38:02,442 - WARN  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:[email protected]] - Cannot open channel to 3 at election address slave-03/202.106.199.35:3888

很显然，slave-01在启动时连接集群中其他结点（slave-02、slave-03）时，主机名映射的IP与我们实际配置的不一致，所以集群中各个结点之间无法建立链路，整个ZooKeeper集群启动是失败的。

上面错误日志中slave-02/202.106.199.35:3888实际应该是slave-02/202.192.168.0.178:3888就对了，但是在进行域名解析的时候映射有问题，修改每个结点的/etc/hosts文件，将ZooKeeper集群中所有结点主机名到IP地址的映射配置上。

时间： 2024-10-14 06:58:28

zookeeper 集群安装（单点与分布式成功安装）摘录的相关文章

zookeeper 集群安装(单点与分布式成功安装)

Cannot open channel to 3 at election address skinglzw2/192.168.128.152:3888 记得关防火墙 centos   service iptables stop

zookeeper 集群安装（单点与分布式成功安装）

参考 http://www.blogjava.net/hello-yun/archive/2012/05/03/377250.html ZooKeeper是一个分布式开源框架,提供了协调分布式应用的基本服务,它向外部应用暴露一组通用服务--分布式同步(Distributed Synchronization).命名服务(Naming Service).集群维护(Group Maintenance)等,简化分布式应用协调及其管理的难度,提供高性能的分布式服务.ZooKeeper本身可以以Standa

Zookeeper集群搭建zookeeper01启动不成功解决方案

查看zookeeper整体启动,zookeeper02,03都启动成功,却只有zookeeper01不起来,纠结了3个小时,重新配置两次,最后发现2181端口被其他进程霸占了,坑啊!!!!! zookeeper01 状态 使用 netstat -atunlp | grep 2181 查看是否霸占进程 发现进程号为2818的java程序霸占了 杀死进程2818 重新启动zookeeper01 启动成功了,功夫不负有心人,继续努力....... 原文地址:https://www.cnblogs.co

ActiveMQ + ZooKeeper 集群高可用配置

一. 准备条件: (1) 最好是有3台服务器[2台也行, 只是根据(replicas/2)+1 公式至少得2个ActiveMQ服务存在才能保证运行, 自己测试的时候麻烦点, 关掉其中一个, 再开启, 看会不会选举到另一个ActiveMQ服务, 多试几次可以看到效果] (2)  ActiveMQ安装参考: ActiveMQ (3)  ZooKeeper安装参考:ZooKeeper 二. 配置 : ActiveMQ根目录下的conf/activemq.xml, 原来默认如下: <persistenc

storm集群和zookeeper集群的部署过程

本次实验的环境为VM station 10 安装三个虚拟机,系统为centOS6.5.IP分别如下图. 如下图,三个虚拟机均安装了Java1.7以及配置好了hosts文件. 接着是下载zookeeper-3.4.6(这里推荐使用此版本),进入conf目录下,将zoo_sample.cfg复制成zoo.cfg,然后修改zoo.cfg里面的参数配置.详细如下截图. 同时,在zookeeper-3.4.6目录中新建data目录.这个data目录一定要新建!!如图. 然后将zookeeper这个文件夹分

使用shell实现zookeeper集群的自动搭建

本例中主节点的主机名为的master,从节点的主机名分别是slave1,slave2,slave3,用户名为hadoop,使用的zookeeper版本为zookeeper3.4.5.在root下运行如下脚本可以实现Zookeeper集群的快速搭建. 在安装之前确保主机之间能够相互通信,修改好主机名,本次试运行一个shall脚本完成3台主机的zookeeper安装. shall脚本代码如下 部署结果如下:

完全分布式ZooKeeper集群和Kafka集群的搭建和使用

自己使用的版本为zookeeper-3.4.7.tar.gz和kafka_2.10-0.9.0.0.tgz.首先要安装JDK(jdk-7u9-linux-i586.tar.gz)和SSH,IP地址的分配为Kafka1(192.168.56.136),Kafka2(192.168.56.137),Kafka3(192.168.56.138).下面主要介绍SSH的安装,ZooKeeper和Kafka集群的搭建和使用. 一. SSH的安装 (1)apt-get install ssh (2)/etc/

【Linux 初学】zookeeper集群、win下Jenkins安装（三）

1.zookeeper集群: Java大型的项目中,环境变量的配置很重要,如果没有很好的配置环境变量的话,甚至项目连启动都是难事. export ZOOKEEPER_HOME=/home/zookeeper-3.3.3 export PATH=$PATH:$ZOOKEEPER_HOME/bin:$ZOOKEEPER_HOME/conf (1)ZooKeeper的单机模式部署 ZooKeeper的单机模式通常是用来快速测试客户端应用程序的,在实际过程中不可能是单机模式.单机模式的配置也比较简单.

ZooKeeper的伪分布式集群搭建以及真分布式集群搭建

zk集群的一些基本概念 zookeeper集群搭建: zk集群,主从节点,心跳机制(选举模式) 配置数据文件 myid 1/2/3 对应 server.1/2/3 通过 zkCli.sh -server [ip]:[port] 命令检测集群是否配置成功 和其他大多数集群结构一样,zookeeper集群也是主从结构.搭建集群时,机器数量最低也是三台,因为小于三台就无法进行选举.选举就是当集群中的master节点挂掉之后,剩余的两台机器会进行选举,在这两台机器中选举出一台来做master节点.而当原