RHEL7/CentOS7 Configure Network Teaming
Network Teaming可联合或合并物理网络接口,形成具有较高吞吐量或冗余方式的逻辑网络接口。其作为网卡绑定的备选方式出现,在系统中和传统的bonding方式并存。
1. 系统环境
[[email protected] ~]# uname -a [[email protected] ~]# uname -a Linux test03 3.10.0-327.el7.x86_64 #1 SMP Thu Nov 19 22:10:57 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux [[email protected] ~]# ip addr show 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: eno16777736: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000 link/ether 00:0c:29:56:69:4d brd ff:ff:ff:ff:ff:ff inet 192.168.231.13/24 brd 192.168.231.255 scope global eno16777736 valid_lft forever preferred_lft forever inet6 fe80::20c:29ff:fe56:694d/64 scope link valid_lft forever preferred_lft forever 3: eno33554992: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000 link/ether 00:0c:29:56:69:57 brd ff:ff:ff:ff:ff:ff 4: eno50332216: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000 link/ether 00:0c:29:56:69:61 brd ff:ff:ff:ff:ff:ff
2. 双网卡绑定目标
- 网卡绑定模式:activebackup - 主备模式
一个网卡处于活跃状态,另一个处于备份状态,所有流量都在主链路上处理,当活跃网卡down掉时,启用备份网卡。
- 绑定网卡:eno33554992+eno50332216=team0
- 设置eno33554992为主网卡(优先处于活跃状态),eno50332216为辅网卡(备份状态,主网卡链路正常时,辅网卡处于备份状态);-- 此时无需设置辅网卡粘滞位。
- 设置eno33554992为主网卡,eno50332216为辅网卡,主网卡链路故障后,辅网卡切换为活跃状态,主网卡链路恢复后,辅网卡仍保持活跃状态直至其链路故障。 -- 此时需要设置辅网卡粘滞位{"sticky" true}。
3. 双网卡绑定步骤
3.1 查看物理网卡信息:
[[email protected] network-scripts]# nmcli device DEVICE TYPE STATE CONNECTION eno16777736 ethernet connected eno16777736 eno33554992 ethernet connected eno33554992 eno50332216 ethernet connected eno50332216 lo loopback unmanaged -- # 共四张网卡,lo为本地回环网卡,另外三张为物理网卡。
3.2 查看网卡连接信息
[[email protected] network-scripts]# nmcli connection show NAME UUID TYPE DEVICE eno50332216 36b38556-4bf8-4ce0-8045-4f5465530f67 802-3-ethernet eno50332216 eno33554992 46831ccf-905b-4361-b5ba-c5448133e288 802-3-ethernet eno33554992 eno16777736 613db14a-2375-4a89-b55a-d2abd8fc65d5 802-3-ethernet eno16777736
3.3 删除网卡连接信息
本次Network Teaming配置中,需要将eno33554992
和eno50332216
绑定为team0
,并且设置eno33554992
为主网卡,首先需要这两块网卡现有的配置信息,否则team0
创建完成后,未删除的网卡配置信息会影响team0
的正常工作。
如果nmcli connection show
命令输出中无将要进行配置的网卡连接信息,则无需进行删除操作。
[[email protected] network-scripts]# nmcli connection delete eno33554992 [[email protected] network-scripts]# nmcli connection delete eno50332216 [[email protected] network-scripts]# [[email protected] network-scripts]# nmcli connection show NAME UUID TYPE DEVICE eno16777736 613db14a-2375-4a89-b55a-d2abd8fc65d5 802-3-ethernet eno16777736 # 网卡连接信息删除成功。 # 这里删除的其实就是/etc/sysconfig/network-scripts目录下两块网卡的配置文件。 [[email protected] network-scripts]# pwd /etc/sysconfig/network-scripts [[email protected] network-scripts]# ls ifcfg-* ifcfg-eno16777736 ifcfg-lo
3.4 创建team0
[[email protected] network-scripts]# nmcli connection add type team ifname team0 con-name team0 config ‘{"runner": {"name":"activebackup"}}‘ ip4 192.168.231.100/24 Connection ‘team0‘ (dfe2f884-40e0-4686-a7f9-2be4b1a4b5b2) successfully added.
设备类型:team;设备名称和连接名称:均为team0;绑定模式:activebackup - 主备模式;IP及子网掩码:192.168.231.100/24;
当然,也可以后续设置IP地址,示例:
#创建team0并设置为主备模式 [[email protected] network-scripts]# nmcli connection add type team ifname team0 con-name team0 config ‘{"runner": {"name":"activebackup"}}‘ #设置IP地址及子网掩码 [[email protected] network-scripts]# nmcli connection modify team0 ipv4.addresses 192.168.231.100/24 #设置IP地址获取方法为手动 [[email protected] network-scripts]# nmcli connection modify team0 ipv4.method manual
配置完成后,会在/etc/sysconfig/network-scripts目录下生成ifcfg-team0的配置文件
[[email protected] network-scripts]# cat /etc/sysconfig/network-scripts/ifcfg-team0 DEVICE=team0 TEAM_CONFIG="{\"runner\": {\"name\":\"activebackup\"}}" DEVICETYPE=Team BOOTPROTO=none DEFROUTE=yes IPV4_FAILURE_FATAL=no IPV6INIT=yes IPV6_AUTOCONF=yes IPV6_DEFROUTE=yes IPV6_FAILURE_FATAL=no NAME=team0 UUID=dfe2f884-40e0-4686-a7f9-2be4b1a4b5b2 ONBOOT=yes IPADDR=192.168.231.100 PREFIX=24 IPV6_PEERDNS=yes IPV6_PEERROUTES=yes
3.5 将两块网卡添加进team0
# 添加网卡eno33554992,设备类型:team-slave;连接名称:team0-p1;master:team0;优先级:100 [[email protected] network-scripts]# nmcli connection add type team-slave ifname eno33554992 con-name team0-p1 master team0 config ‘{"prio": 100}‘ Connection ‘team0-p1‘ (4c8901c7-0246-47ce-8d5a-272f23f88d70) successfully added. # 添加网卡eno50332216,设备类型:team-slave;连接名称:team0-p1;master:team0;优先级:-10; 粘滞位:true(sticky标志请根据实际需要设置) [[email protected] network-scripts]# nmcli connection add type team-slave ifname eno50332216 con-name team0-p2 master team0 config ‘{"prio": -10,"sticky": true}‘ Connection ‘team0-p2‘ (d898f1f5-bb0f-496e-8cdd-7f3898c2a482) successfully added.
本次配置中
eno33554992
为主网卡,设置其优先级为100;eno50332216
为辅网卡,设置其优先级为-10,粘滞位为true。prio:优先级。
数字越大,网卡优先级越高,优先为活跃网卡;若不设置优先级,则默认为0,先接入网络的网卡为活跃网卡,另一块为备份网卡,直至活跃网卡断开连接,备份网卡自动升级为活跃网卡,两块网卡角色互换。
sticky:粘滞位,默认为false;
- 辅网卡
eno50332216
中设置了sticky为true,代表如果辅网卡连接状态变为为活跃状态后,无论主网卡连接是否正常,其将会一直保持为活跃状态。- 若不设置sticky,在辅网卡为活跃状态时,若主网卡连接恢复正常,则活跃网卡由主网卡接管,辅网卡会变为备份状态。
可根据实际需要进行参数设置。
配置完成后,此时会在/etc/sysconfig/network-scripts目录下生成ifcfg-team0-p1,ifcfg-team0-p2的配置文件:
[[email protected] network-scripts]# cat ifcfg-team0-p1 TEAM_PORT_CONFIG="{\"prio\": 100}" NAME=team0-p1 UUID=4c8901c7-0246-47ce-8d5a-272f23f88d70 DEVICE=eno33554992 ONBOOT=yes TEAM_MASTER=team0 DEVICETYPE=TeamPort [[email protected] network-scripts]# cat ifcfg-team0-p2 TEAM_PORT_CONFIG="{\"prio\": -10,\"sticky\": true}" NAME=team0-p2 UUID=d898f1f5-bb0f-496e-8cdd-7f3898c2a482 DEVICE=eno50332216 ONBOOT=yes TEAM_MASTER=team0 DEVICETYPE=TeamPort
4. 激活team0及其子网卡
# 激活team0-p1 [[email protected] ~]# nmcli connection up team0-p1 Connection successfully activated (D-Bus active path: /org/freedesktop/NetworkManager/ActiveConnection/5) # 激活team0-p2 [[email protected] ~]# nmcli connection up team0-p2 Connection successfully activated (D-Bus active path: /org/freedesktop/NetworkManager/ActiveConnection/6) # 激活team0 [[email protected] ~]# nmcli connection up team0 Connection successfully activated (master waiting for slaves) (D-Bus active path: /org/freedesktop/NetworkManager/ActiveConnection/7)
5. team0状态查看
5.1 查看team0工作状态:
# 查看当前活跃网卡 [[email protected] network-scripts]# nmcli connection show --active NAME UUID TYPE DEVICE team0-p1 4c8901c7-0246-47ce-8d5a-272f23f88d70 802-3-ethernet eno33554992 eno16777736 613db14a-2375-4a89-b55a-d2abd8fc65d5 802-3-ethernet eno16777736 team0 d6e07840-dff8-49e9-a23c-35eb0cc0ec4b team team0 team0-p2 d898f1f5-bb0f-496e-8cdd-7f3898c2a482 802-3-ethernet eno50332216 #查看team0端口状态 [[email protected] network-scripts]# teamnl team0 ports 4: eno50332216: up 1000Mbit FD 3: eno33554992: up 1000Mbit FD [[email protected] network-scripts]# teamdctl team0 stat setup: runner: activebackup ports: eno33554992 link watches: link summary: up instance[link_watch_0]: name: ethtool link: up eno50332216 link watches: link summary: up instance[link_watch_0]: name: ethtool link: up runner: active port: eno33554992
5.2 查看team0详细配置
# 查看team0详细配置 [[email protected] network-scripts]# teamdctl team0 config dump { "device": "team0", "mcast_rejoin": { "count": 1 }, "notify_peers": { "count": 1 }, "ports": { "eno33554992": { "link_watch": { "name": "ethtool" }, "prio": 100 }, "eno50332216": { "link_watch": { "name": "ethtool" }, "prio": -10, "sticky": true } }, "runner": { "name": "activebackup" } } # 单独查看team0子网卡详细配置 [[email protected] network-scripts]# teamdctl team0 port config dump eno33554992 { "link_watch": { "name": "ethtool" }, "prio": 100 } [[email protected] network-scripts]# teamdctl team0 port config dump eno50332216 { "link_watch": { "name": "ethtool" }, "prio": -10, "sticky": true }
6. team0切换测试
[[email protected] ~]# teamdctl team0 stat setup: runner: activebackup ports: eno33554992 link watches: link summary: up instance[link_watch_0]: name: ethtool link: up eno50332216 link watches: link summary: up instance[link_watch_0]: name: ethtool link: up runner: active port: eno33554992
当前活跃子网卡为eno33554992
,ping team0的IP地址一直可以ping通。现在将该网卡断开连接:
# 断开team0-p1的网络连接 [[email protected] ~]# nmcli connection down team0-p1 Connection ‘team0-p1‘ successfully deactivated (D-Bus active path: /org/freedesktop/NetworkManager/ActiveConnection/2) # 查看team0连接状态 [[email protected] ~]# teamdctl team0 stat setup: runner: activebackup ports: eno50332216 link watches: link summary: up instance[link_watch_0]: name: ethtool link: up runner: active port: eno50332216 # 活跃网卡变为eno50332216,ping测试短暂中断后恢复。 来自 192.168.231.100 的回复: 字节=32 时间<1ms TTL=64 来自 192.168.231.100 的回复: 字节=32 时间<1ms TTL=64 来自 192.168.231.100 的回复: 字节=32 时间=1ms TTL=64 来自 192.168.231.100 的回复: 字节=32 时间<1ms TTL=64 请求超时。 请求超时。 来自 192.168.231.100 的回复: 字节=32 时间=1ms TTL=64 来自 192.168.231.100 的回复: 字节=32 时间<1ms TTL=64 来自 192.168.231.100 的回复: 字节=32 时间<1ms TTL=64
由于本次测试在虚拟机中进行,ping测试会出现较明显的中断,在真实的物理机上测试时,ping测试会一直正常或出现短暂中断后恢复。
官方文档推荐使用:
nmcli device [disconnect|connect] ifname
命令来进行网卡的断开和连接操作,但在虚拟机上使用此方法断开网卡连接后,恢复网卡连接时会报错,这里使用命令nmcli connection [down|up] ifname
进行操作。
当前活跃网卡为辅网卡eno50332216
,现在连接主网卡:
# 恢复team0-p1的网络连接 [[email protected] ~]# nmcli connection up team0-p1 Connection successfully activated (D-Bus active path: /org/freedesktop/NetworkManager/ActiveConnection/7) # 查看team0的连接状态 [[email protected] ~]# teamdctl team0 stat setup: runner: activebackup ports: eno33554992 link watches: link summary: up instance[link_watch_0]: name: ethtool link: up eno50332216 link watches: link summary: up instance[link_watch_0]: name: ethtool link: up runner: active port: eno50332216 # ping测试短暂中断后恢复 来自 192.168.231.100 的回复: 字节=32 时间<1ms TTL=64 请求超时。 请求超时。 来自 192.168.231.100 的回复: 字节=32 时间=1ms TTL=64 来自 192.168.231.100 的回复: 字节=32 时间<1ms TTL=64
由于本次测试在虚拟机中进行,ping测试会出现较明显的中断,在真实的物理机上测试时,ping测试会一直正常或出现短暂中断后恢复。
在主网卡连接恢复正常后,查看team0发现当前活跃网卡仍为辅网卡
eno50332216
,没有将活跃网卡切换为主网卡eno33554992
,原因是在辅网卡中设置了粘滞位sticky=true,sticky标志使辅网卡为活跃状态后,若其网络连接不发生故障,则会一直保持其为活跃网卡。如果在辅网卡配置中不设置sticky标志,主网卡连接恢复正常后,活跃网卡由主网卡接管,辅网卡会变为备份状态。
7. 双网卡绑定各模式配置文件示例
对于各种双网卡绑定配置模式,请参考系统中的示例文件,文件位置:
/usr/share/doc/teamd-xx/example_configs/