ONOS HA 部署流程
高可靠性的网络环境在生产中随处可见，ONOS的一大亮点是支持集群。为了能够快速的搭建起ONOS的集群环境，官网提供了基于docker的部署镜像。当我们部署完成docker环境以后，可以很方便的使用docker来部署一个ONOS的集群。[关于docker的环境部署，参考：http://www.cnblogs.com/BurlyLuo/p/8283815.html]
ONOS使用Hazelcast架构实现对集群成员的管理，在集群模式下，网络中的控制器共享状态信息，共同管理网络中的设备（交换机）。使用ONOS建立集群的时候会将集群中的设备都默认加入到224.2.2.3:54327组播组中，组播地址可以通过修改hazelcast.xml文件指定。
1.基础平台搭建。
本实验是在上一篇博客[SDN模拟器ONOS Install Guide]中提到的ONOS环境中利用容器技术做出的另一套ONOS的集群环境，CPU和memory配置均未更改，发现运行没有任何问题。我在实际中加了一块桥接网卡，配置的地址：192.168.43.119//方便通过CRT软件登录实现copy/paste//可不做，直接在图形化桌面操作即可。//但是必须的使用distributed/distributed 用户名/密码登录CLI，否则会导致部分命令无法使用。//Like This:----------------------------------------------------------------------Last login: Mon Feb 19 18:20:56 2018 from luowei[email protected]:~$ ssh [email protected] //用户名：distributed[email protected]‘s password: //密码为：distributedWelcome to Ubuntu 14.10 (GNU/Linux 3.16.0-23-generic x86_64) * Documentation: https://help.ubuntu.com/Your Ubuntu release is not supported anymore.For upgrade information, please visit:http://www.ubuntu.com/releaseendoflife
2.使用docker部署ONOS集群
2.1查看该系统中的docker images，如果没有可以使用docker pull sdnsub/onos 来拉取对应的ONOS image。[email protected]:~$ sudo docker imagesREPOSITORY TAG IMAGE ID CREATED VIRTUAL SIZEonos/tutorial-dist latest cc5df7fe591e 3.141949 years ago 661.5 MBubuntu-upstart 14.10 ac87bc62880a 3.184969 years ago 259.4 MB[email protected]:~$ 2.2在该系统上运行三个docker container。[email protected]:~$ sudo docker run -t -P -i -d --name onos-1 onos/tutorial-dist[email protected]:~$ sudo docker run -t -P -i -d --name onos-2 onos/tutorial-dist[email protected]:~$ sudo docker run -t -P -i -d --name onos-3 onos/tutorial-dist查看运行的container:[email protected]:~$ sudo docker ps -aCONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMESb5f408276c70 onos/tutorial-dist:latest "/sbin/init" 22 minutes ago Up 22 minutes 0.0.0.0:49159->22/tcp, 0.0.0.0:49160->6633/tcp, 0.0.0.0:49161->8181/tcp onos-3 34e068d7032b onos/tutorial-dist:latest "/sbin/init" 22 minutes ago Up 22 minutes 0.0.0.0:49156->22/tcp, 0.0.0.0:49157->6633/tcp, 0.0.0.0:49158->8181/tcp onos-2 1f3a3cdc3b63 onos/tutorial-dist:latest "/sbin/init" 23 minutes ago Up 23 minutes 0.0.0.0:49153->22/tcp, 0.0.0.0:49154->6633/tcp, 0.0.0.0:49155->8181/tcp onos-1 2.3检查docker的环境是否正确。[email protected]:~$ cell docker ONOS_CELL=dockerOCI=172.17.0.2OC1=172.17.0.2OC2=172.17.0.3OC3=172.17.0.4OCN=localhostONOS_FEATURES=webconsole,onos-api,onos-core,onos-cli,onos-rest,onos-gui,onos-openflow,onos-app-fwd,onos-app-proxyarp,onos-app-mobilityONOS_USER=rootONOS_NIC=172.17.0.*-------------------------------------------------------------------------------------------------------------------------------------------该步骤非常关键。跳过可能会导致$OC的值没有及时更新而发生错误。我们知道在docker默认的情况下，docker0为172.17.0.0/16的网段，所以默认情况下刚才创建的三个containers所获得地址为172.17.0.x的地址。我们可以进入container中查看地址。-------------------------------------------------------------------------------------------------------------------------------------------2.4设置无密码访问系统镜像提供了该脚本，我们可以直接使用。[其中默认的密码为：onosrocks][email protected]:~$ onos-push-keys $OC1The authenticity of host ‘172.17.0.2 (172.17.0.2)‘ can‘t be established.ECDSA key fingerprint is a7:83:a2:2e:43:e3:39:19:1f:a5:05:ae:80:74:26:b6.Are you sure you want to continue connecting (yes/no)? yes[email protected]‘s password: [email protected]‘s password: [email protected]:~$ onos-push-keys $OC2The authenticity of host ‘172.17.0.3 (172.17.0.3)‘ can‘t be established.ECDSA key fingerprint is a7:83:a2:2e:43:e3:39:19:1f:a5:05:ae:80:74:26:b6.Are you sure you want to continue connecting (yes/no)? yes[email protected]‘s password: [email protected]‘s password: [email protected]:~$ onos-push-keys $OC3The authenticity of host ‘172.17.0.4 (172.17.0.4)‘ can‘t be established.ECDSA key fingerprint is a7:83:a2:2e:43:e3:39:19:1f:a5:05:ae:80:74:26:b6.Are you sure you want to continue connecting (yes/no)? yes[email protected]‘s password: [email protected]‘s password:2.5下载对应的onos软件包[email protected]:~$ onos-package-rw-rw-r-- 1 distributed distributed 41940635 Feb 19 18:07 /tmp/onos-1.0.0.distributed.tar.gz2.6在刚才部署的三个docker container中安装onos。[email protected]:~$ onos-install $OC1onos start/running, process 315[email protected]:~$ onos-install $OC2onos start/running, process 317[email protected]:~$ onos-install $OC3onos start/running, process 3152.7进入onos的cliFor OC1
--------------------------------------------------------------ONOS-1--------------------------------------------------------------[email protected]:~$ onos -w $OC1Connection to 172.17.0.2 closed.client: JAVA_HOME not set; results may varyLogging in as karafWelcome to Open Network Operating System (ONOS)! ____ _ ______ ____ / __ \/ |/ / __ \/ __/ / /_/ / / /_/ /\ \ \____/_/|_/\____/___/ Hit ‘<tab>‘ for a list of available commandsand ‘[cmd] --help‘ for help on a specific command.Hit ‘<ctrl-d>‘ or type ‘system:shutdown‘ or ‘logout‘ to shutdown ONOS.For OC2--------------------------------------------------------------ONOS-2--------------------------------------------------------------Last login: Mon Feb 19 17:14:09 2018 from luowei[email protected]:~$ [email protected]:~$ ssh distributed192.168.43.19^C[email protected]:~$ ssh distributed192.168.43.119^C[email protected]:~$ ssh [email protected][email protected]‘s password: Welcome to Ubuntu 14.10 (GNU/Linux 3.16.0-23-generic x86_64) * Documentation: https://help.ubuntu.com/Your Ubuntu release is not supported anymore.For upgrade information, please visit:http://www.ubuntu.com/releaseendoflife New release ‘15.04‘ available.Run ‘do-release-upgrade‘ to upgrade to it. Last login: Mon Feb 19 18:17:00 2018 from 192.168.43.119[email protected]:~$ onos -w $OC2Connection to 172.17.0.3 closed.client: JAVA_HOME not set; results may varyLogging in as karafWelcome to Open Network Operating System (ONOS)! ____ _ ______ ____ / __ \/ |/ / __ \/ __/ / /_/ / / /_/ /\ \ \____/_/|_/\____/___/ Hit ‘<tab>‘ for a list of available commandsand ‘[cmd] --help‘ for help on a specific command.Hit ‘<ctrl-d>‘ or type ‘system:shutdown‘ or ‘logout‘ to shutdown ONOS. onos> For OC3------------------------------------------------------------ONOS-3------------------------------------------------------------Last login: Mon Feb 19 18:11:07 2018 from luowei[email protected]:~$ ssh [email protected]ssh: connect to host 192.168.43.19 port 22: No route to host[email protected]:~$ ssh [email protected]The authenticity of host ‘192.168.43.119 (192.168.43.119)‘ can‘t be established.ECDSA key fingerprint is b2:89:b0:31:07:05:f3:28:88:be:19:71:79:9a:f5:21.Are you sure you want to continue connecting (yes/no)? yesWarning: Permanently added ‘192.168.43.119‘ (ECDSA) to the list of known hosts.[email protected]‘s password: Welcome to Ubuntu 14.10 (GNU/Linux 3.16.0-23-generic x86_64) * Documentation: https://help.ubuntu.com/Your Ubuntu release is not supported anymore.For upgrade information, please visit:http://www.ubuntu.com/releaseendoflife New release ‘15.04‘ available.Run ‘do-release-upgrade‘ to upgrade to it. Last login: Mon Feb 19 18:04:16 2018 from 192.168.43.119[email protected]:~$ onos -w $OC3Connection to 172.17.0.4 closed.client: JAVA_HOME not set; results may varyLogging in as karafWelcome to Open Network Operating System (ONOS)! ____ _ ______ ____ / __ \/ |/ / __ \/ __/ / /_/ / / /_/ /\ \ \____/_/|_/\____/___/ Hit ‘<tab>‘ for a list of available commandsand ‘[cmd] --help‘ for help on a specific command.Hit ‘<ctrl-d>‘ or type ‘system:shutdown‘ or ‘logout‘ to shutdown ONOS. onos> 至此SND的控制器ONOS已经部署完成，有了前边的经验，这里有可能需要自己安装一些feature，比如ONOS GUI的feature等等，需要特别的注意。
3.启动MIninet。
[email protected]:~$ ssh [email protected] [email protected]‘s password: Welcome to Ubuntu 14.10 (GNU/Linux 3.16.0-23-generic x86_64) * Documentation: https://help.ubuntu.com/ Your Ubuntu release is not supported anymore. For upgrade information, please visit: http://www.ubuntu.com/releaseendoflife New release ‘15.04‘ available. Run ‘do-release-upgrade‘ to upgrade to it. Last login: Mon Feb 19 18:17:10 2018 from 192.168.43.119 [email protected]:~$ cd onos-byon && ./startmn.sh *** Creating network *** Adding hosts: h11 h12 h13 h14 h15 h16 h21 h22 h23 h24 h25 h26 h31 h32 h33 h34 h35 h36 h41 h42 h43 h44 h45 h46 *** Adding switches: s1 s2 s11 s12 s13 s14 *** Adding links: (h11, s11) (h12, s11) (h13, s11) (h14, s11) (h15, s11) (h16, s11) (h21, s12) (h22, s12) (h23, s12) (h24, s12) (h25, s12) (h26, s12) (h31, s13) (h32, s13) (h33, s13) (h34, s13) (h35, s13) (h36, s13) (h41, s14) (h42, s14) (h43, s14) (h44, s14) (h45, s14) (h46, s14) (s1, s2) (s11, s1) (s11, s2) (s12, s1) (s12, s2) (s13, s1) (s13, s2) (s14, s1) (s14, s2) *** Configuring hosts h11 h12 h13 h14 h15 h16 h21 h22 h23 h24 h25 h26 h31 h32 h33 h34 h35 h36 h41 h42 h43 h44 h45 h46 *** Starting controller c0 c0 c0 *** Starting 6 switches s1 s2 s11 s12 s13 s14 *** Starting CLI: mininet> mininet> pingall *** Ping: testing ping reachability h11 -> h12 h13 h14 h15 h16 h21 h22 h23 h24 h25 h26 h31 h32 h33 h34 h35 h36 h41 h42 h43 h44 h45 h46 h12 -> h11 h13 h14 h15 h16 h21 h22 h23 h24 h25 h26 h31 h32 h33 h34 h35 h36 h41 h42 h43 h44 h45 h46 h13 -> h11 h12 h14 h15 h16 h21 h22 h23 h24 h25 h26 h31 h32 h33 h34 h35 h36 h41 h42 h43 h44 h45 h46 h14 -> h11 h12 h13 h15 h16 h21 h22 h23 h24 h25 h26 h31 h32 h33 h34 h35 h36 h41 h42 h43 h44 h45 h46 h15 -> h11 h12 h13 h14 h16 h21 h22 h23 h24 h25 h26 h31 h32 h33 h34 h35 h36 h41 h42 h43 h44 h45 h46 h16 -> h11 h12 h13 h14 h15 h21 h22 h23 h24 h25 h26 h31 h32 h33 h34 h35 h36 h41 h42 h43 h44 h45 h46 h21 -> h11 h12 h13 h14 h15 h16 h22 h23 h24 h25 h26 h31 h32 h33 h34 h35 h36 h41 h42 h43 h44 h45 h46 h22 -> h11 h12 h13 h14 h15 h16 h21 h23 h24 h25 h26 h31 h32 h33 h34 h35 h36 h41 h42 h43 h44 h45 h46 h23 -> h11 h12 h13 h14 h15 h16 h21 h22 h24 h25 h26 h31 h32 h33 h34 h35 h36 h41 h42 h43 h44 h45 h46 h24 -> h11 h12 h13 h14 h15 h16 h21 h22 h23 h25 h26 h31 h32 h33 h34 h35 h36 h41 h42 h43 h44 h45 h46 h25 -> h11 h12 h13 h14 h15 h16 h21 h22 h23 h24 h26 h31 h32 h33 h34 h35 h36 h41 h42 h43 h44 h45 h46 h26 -> h11 h12 h13 h14 h15 h16 h21 h22 h23 h24 h25 h31 h32 h33 h34 h35 h36 h41 h42 h43 h44 h45 h46 h31 -> h11 h12 h13 h14 h15 h16 h21 h22 h23 h24 h25 h26 h32 h33 h34 h35 h36 h41 h42 h43 h44 h45 h46 h32 -> h11 h12 h13 h14 h15 h16 h21 h22 h23 h24 h25 h26 h31 h33 h34 h35 h36 h41 h42 h43 h44 h45 h46 h33 -> h11 h12 h13 h14 h15 h16 h21 h22 h23 h24 h25 h26 h31 h32 h34 h35 h36 h41 h42 h43 h44 h45 h46 h34 -> h11 h12 h13 h14 h15 h16 h21 h22 h23 h24 h25 h26 h31 h32 h33 h35 h36 h41 h42 h43 h44 h45 h46 h35 -> h11 h12 h13 h14 h15 h16 h21 h22 h23 h24 h25 h26 h31 h32 h33 h34 h36 h41 h42 h43 h44 h45 h46 h36 -> h11 h12 h13 h14 h15 h16 h21 h22 h23 h24 h25 h26 h31 h32 h33 h34 h35 h41 h42 h43 h44 h45 h46 h41 -> h11 h12 h13 h14 h15 h16 h21 h22 h23 h24 h25 h26 h31 h32 h33 h34 h35 h36 h42 h43 h44 h45 h46 h42 -> h11 h12 h13 h14 h15 h16 h21 h22 h23 h24 h25 h26 h31 h32 h33 h34 h35 h36 h41 h43 h44 h45 h46 h43 -> h11 h12 h13 h14 h15 h16 h21 h22 h23 h24 h25 h26 h31 h32 h33 h34 h35 h36 h41 h42 h44 h45 h46 h44 -> h11 h12 h13 h14 h15 h16 h21 h22 h23 h24 h25 h26 h31 h32 h33 h34 h35 h36 h41 h42 h43 h45 h46 h45 -> h11 h12 h13 h14 h15 h16 h21 h22 h23 h24 h25 h26 h31 h32 h33 h34 h35 h36 h41 h42 h43 h44 h46 h46 -> h11 h12 h13 h14 h15 h16 h21 h22 h23 h24 h25 h26 h31 h32 h33 h34 h35 h36 h41 h42 h43 h44 h45 *** Results: 0% dropped (552/552 received) mininet> h12 ping h13 PING 10.0.0.3 (10.0.0.3) 56(84) bytes of data. 64 bytes from 10.0.0.3: icmp_seq=1 ttl=64 time=20.8 ms 64 bytes from 10.0.0.3: icmp_seq=2 ttl=64 time=0.573 ms 64 bytes from 10.0.0.3: icmp_seq=3 ttl=64 time=0.064 ms ^C --- 10.0.0.3 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 2005ms rtt min/avg/max/mdev = 0.064/7.157/20.835/9.674 ms mininet> mininet> nodes available nodes are: c0 c0 c0 h11 h12 h13 h14 h15 h16 h21 h22 h23 h24 h25 h26 h31 h32 h33 h34 h35 h36 h41 h42 h43 h44 h45 h46 s1 s11 s12 s13 s14 s2 mininet>
4.安装完ONOS的feature之后，我们可以在浏览器中打开部署好的环境。
在ONOS-1上查看 onos> summary node=172.17.0.2, version=1.0.0.distributed~2018/02/[email protected]:06 nodes=3, devices=0, links=0, hosts=0, SCC(s)=0, paths=0, flows=0, intents=0 onos> masters 172.17.0.2: 0 devices 172.17.0.3: 0 devices 172.17.0.4: 0 devices onos> devices onos> roles onos> 激活Mininit以后： onos> devices id=of:0000000000000001, available=true, role=MASTER, type=SWITCH, mfr=Nicira, Inc., hw=Open vSwitch, sw=2.1.3, serial=None, protocol=OF_10 id=of:0000000000000002, available=true, role=MASTER, type=SWITCH, mfr=Nicira, Inc., hw=Open vSwitch, sw=2.1.3, serial=None, protocol=OF_10 id=of:000000000000000b, available=true, role=MASTER, type=SWITCH, mfr=Nicira, Inc., hw=Open vSwitch, sw=2.1.3, serial=None, protocol=OF_10 id=of:000000000000000c, available=true, role=STANDBY, type=SWITCH, mfr=Nicira, Inc., hw=Open vSwitch, sw=2.1.3, serial=None, protocol=OF_10 id=of:000000000000000d, available=true, role=STANDBY, type=SWITCH, mfr=Nicira, Inc., hw=Open vSwitch, sw=2.1.3, serial=None, protocol=OF_10 id=of:000000000000000e, available=true, role=STANDBY, type=SWITCH, mfr=Nicira, Inc., hw=Open vSwitch, sw=2.1.3, serial=None, protocol=OF_10 onos> onos> balance-masters //做负载均衡// onos> masters 172.17.0.2: 2 devices of:0000000000000002 of:000000000000000b 172.17.0.3: 2 devices of:000000000000000c of:000000000000000e 172.17.0.4: 2 devices of:0000000000000001 of:000000000000000d onos> onos> nodes id=172.17.0.2, address=172.17.0.2:9876, state=ACTIVE * id=172.17.0.3, address=172.17.0.3:9876, state=ACTIVE id=172.17.0.4, address=172.17.0.4:9876, state=ACTIVE
当然这里主要是针对于集群环境下的部署的一个说明，如果我们停掉一个ONOS节点 ，我们会在ONOS的GUi上看到相应的控制器退出集群，剩下的继续提供服务。可自己实验，这里就不做赘述。
这里顺便提一下ONOS和ODL的比较。节选自知乎用户:常景路.
ONOS和ODL分别由运营商和厂商主导，所代表的利益不同，也就分别选择了两种不同的SDN演进方式。前者更贴近于SDN诞生之初时狭义的SDN概念，即通过OpenFlow将控制平面和转发平面完全分离，网络设备只是进行转发的黑盒子，通过Controller完成一切计算。ONOS所选择的理念与运营商自己的利益息息相关，只有将控制能力拿到自己手里，才能在整条产业链上逐步摆脱设备厂商的控制。通过使用更为廉价的转发设备替代原有的厂商设备，一方面在眼下增加自己与设备厂商的议价砝码，另一方面长远看能大大降低网络的建设和维护成本。相比较而言，ODL则采取了更为平缓的SDN演进方式，从理念上更为贴近广义的SDN，即不局限于OpenFlow协议，不局限于完全将控制平面从转发设备上剥离，通过已有的网络协议将部分的控制逻辑放到Controller上。这样的理念使广义的SDN技术的落地更容易成为现实，一方面通过保护运营商、企业等设备厂商客户的既有投资，使客户可以真正感受到SDN技术的实际效果。另一方面，通过在现有设备上扩展已有的网络协议，厂商能够使自己的设备在不用伤筋动骨就能保有竞争力，避免自己在SDN的革命中被迅速甩下。从技术上讲，SDN Controller实际上解决的是南向与设备的通信问题和北向向APP提供的资源问题，网络运营者根据自己网络的业务特点提出的控制逻辑则需要开发APP来实现。从南向接口上看，ONOS目前成熟的南向接口只有OpenFlow，而ODL Helium版则支持OpenFlow、OVS-DB、MP-BGP、PCEP、NETCONF/YANG等极为丰富的南向接口以连接不同类型的设备。从北向接口上看，ODL采用的MD-SAL使得设备资源可以通过YANG model直接转换为RESTConf API，而ONOS还在某种程度上停留在ODL最初版本使用的AD-SAL架构，API需要在plugin设计时单独考量。当然除此之外，Controller的性能与Scale out也是必须面对的问题。对此，ONOS确实抓住了ODL尚未解决的问题，从一开始就从这两方面抢占先机，拨人眼球。不过从二者实现上都采用了JAVA的Karaf框架来看，性能与Scale out问题在根本上也不会存在先天的差别，面对海量计算采用Cluster会是最终的解决方法，而实际上两个控制器都提供了相应的Cluster部署方案。唯一的问题可能是ODL还需要应对多种南向接口带来的额外消耗，但ODL提供的是南向接口的可选能力，实际部署上也很少会出现多种协议共存的情况。
5.如有疑问。
######################################################################## #Date:2018-03-08 Author:BurlyLuo# #Mail:[email protected] Version:v1.0 # ########################################################################非商业用途转载无需作者授权，但务必在文章标题下面注明作者 罗伟（Burly Luo）以及可点击的本博客地址超级链接 http://www.cnblogs.com/BurlyLuo/，谢谢合作.
原文地址：https://www.cnblogs.com/BurlyLuo/p/8531019.html
时间： 2024-11-07 03:34:59