Azure上有IPSec VPN Gateway的服务，VPN是在公有云服务中客户最常使用的网络服务之一。
Azure的IPSec VPN Gateway可以支持IKEv1和IKEv2两种，之前的文章中已经做了一些介绍。同时，Azure IPSec VPN还可以和专线的ER Gateway共存。这个话题之前也讨论过多次。感兴趣的话，可以看以前的文章。
但Azure之前的IPSec VPN的服务中，所有的模式都是静态的方式，不支持动态路由。目前Azure China的IPSec VPN Gateway已经支持BGP动态路由协议了。
本文将介绍支持BGP的IPSec VPN Gateway的使用场景，已经具体的配置。
一、Azure IPSec VPN Gateway支持BGP的使用场景
1. BGP
BGP是Border Gateway Protocol的简称，是企业、运营商等大型网络进行网络互联时采用的动态路由协议。BGP本身比较复杂，本文就不再展开讨论了。BGP的典型部署场景如下：
在IPSec VPN Gateway支持了BGP路由协议后，各个VNET和用户侧的网络可以动态的学习路由，实现路由的动态学习和调整，从而实现冗余、备份等功能。
2. 在Azure的Vnet环境下常用的场景
由于BGP可以实现动态路由，可以用BGP来实现冗余和备份。
三个Vnet间采用BGP互联，可以实现冗余和备份。Vnet1、Vnet2、Vnet3的路由信息通过BGP进行交换。备份的实现：比如Vnet1和Vnet2的连接，在正常情况下通过Vnet1和Vnet2间的IPSec进行通信，当这根链路出现故障时，流量从Vnet1-Vnet3-Vnet2的链路传输。
一个VPN Gateway可以和两个OnPrem的VPN Gateway建立IPSec的连接，路由信息通过BGP进行交换，当一个GW或运营商链路出现故障时，不影响Azure和OnPrem的VPN连接。
二、BGP的具体配置
1. 前文提到的第一种场景的三个Vnet间的IPSec + BGP的互联：
a. 创建了3个Vnet：
每个Vnet分配不同的地址空间，并配置相应的Subnet，和Gateway Subnet。
b. 在每个Vnet中创建VPN Gateway，类型选择Standard。
三个Vnet的网段信息：
$vnets = Get-AzureRmVirtualNetwork -ResourceGroupName vpn
foreach ($vnet in $vnets){
Write-Host $vnet.Name $vnet.AddressSpace.AddressPrefixes
foreach ($i in @(0,1)){
write-host " " $vnet.Subnets[$i].name $vnet.Subnets[$i].addressprefix
}
}
vnet1 10.1.0.0/16
vlan1 10.1.0.0/24
GatewaySubnet 10.1.254.240/28
vnet2 10.2.0.0/16
vlan2 10.2.0.0/24
GatewaySubnet 10.2.254.240/28
vnet3 10.3.0.0/16
vlan3 10.3.0.0/24
GatewaySubnet 10.3.254.240/28
通过powershell命令：
$vpngws = Get-AzureRmVirtualNetworkGateway -ResourceGroupName vpn
foreach ($vpngw in $vpngws){
write-host $vpngw.name $vpngw.sku.Tier $vpngw.VpnType $vpngw.EnableBgp
}
vnet1gw Standard RouteBased False
vnet2gw Standard RouteBased False
vnet3gw Standard RouteBased False
可以看到，三个Vnet中的VPN Gateway的名称、型号、类型和是否支持BGP。
目前这三个VPN Gateway都不支持BGP。
foreach ($vpngw in $vpngws){
write-host $vpngw.Name
Write-Host " " $vpngw.BgpSettingsText
}
vnet1gw
{
"Asn": 65515,
"BgpPeeringAddress": "10.1.254.254",
"PeerWeight": 0
}
vnet2gw
{
"Asn": 65515,
"BgpPeeringAddress": "10.2.254.254",
"PeerWeight": 0
}
vnet3gw
{
"Asn": 65515,
"BgpPeeringAddress": "10.3.254.254",
"PeerWeight": 0
}
c. 在VPN GW上配置BGP参数
下面我们对VPN Gateway的属性进行修改：
$i
=
0
foreach ($vpngw
in
$vpngws){
$i
=
$i
+
1
$vpngw.EnableBgp = $true
$vpngw.BgpSettings.Asn =
65000
+ $i
}
vnet1gw
{
"Asn": 65001,
"BgpPeeringAddress": "10.1.254.254",
"PeerWeight": 0
}
vnet2gw
{
"Asn": 65002,
"BgpPeeringAddress": "10.2.254.254",
"PeerWeight": 0
}
vnet3gw
{
"Asn": 65003,
"BgpPeeringAddress": "10.3.254.254",
"PeerWeight": 0
}
vnet1gw Standard RouteBased True
vnet2gw Standard RouteBased True
vnet3gw Standard RouteBased True
设置VPN Gateway：
foreach ( $vpngw
in
$vpngws){
Set-AzureRmVirtualNetworkGateway
-VirtualNetworkGateway
$vpngw
}
foreach ( $vpngw in $vpngws){
Set-AzureRmVirtualNetworkGateway -VirtualNetworkGateway $vpngw
}
Name : vnet1gw
ResourceGroupName : vpn
Location : chinaeast
Id : /subscriptions/20311952-a4a1-4ab2-a434-a71269a2dd52/resourceGroups/vpn/providers/Microsoft.Network/virtualNet
workGateways/vnet1gw
Etag : W/"968c154b-03a2-4251-9c2d-c9da1d285721"
ResourceGuid : 1b222fc2-5e25-41e8-b79a-cd110fc91ec1
ProvisioningState : Succeeded
Tags :
IpConfigurations : [
{
"PrivateIpAllocationMethod": "Dynamic",
"Subnet": {
"Id": "/subscriptions/20311952-a4a1-4ab2-a434-a71269a2dd52/resourceGroups/vpn/providers/Microsoft.Netwo
rk/virtualNetworks/vnet1/subnets/GatewaySubnet"
},
"PublicIpAddress": {
"Id": "/subscriptions/20311952-a4a1-4ab2-a434-a71269a2dd52/resourceGroups/vpn/providers/Microsoft.Netwo
rk/publicIPAddresses/vnet1gwip"
},
"Name": "default",
"Etag": "W/\"968c154b-03a2-4251-9c2d-c9da1d285721\"",
"Id": "/subscriptions/20311952-a4a1-4ab2-a434-a71269a2dd52/resourceGroups/vpn/providers/Microsoft.Network
/virtualNetworkGateways/vnet1gw/ipConfigurations/default"
}
]
GatewayType : Vpn
VpnType : RouteBased
EnableBgp : True
ActiveActive : False
GatewayDefaultSite : null
Sku : {
"Capacity": 2,
"Name": "Standard",
"Tier": "Standard"
}
VpnClientConfiguration : null
BgpSettings : {
"Asn": 65001,
"BgpPeeringAddress": "10.1.254.254",
"PeerWeight": 0
}
Name : vnet2gw
ResourceGroupName : vpn
Location : chinaeast
Id : /subscriptions/20311952-a4a1-4ab2-a434-a71269a2dd52/resourceGroups/vpn/providers/Microsoft.Network/virtualNet
workGateways/vnet2gw
Etag : W/"af09f17a-88b9-42d7-ab51-c512062430f6"
ResourceGuid : a4b25b2d-9c88-48f2-a669-24b177e57920
ProvisioningState : Succeeded
Tags :
IpConfigurations : [
{
"PrivateIpAllocationMethod": "Dynamic",
"Subnet": {
"Id": "/subscriptions/20311952-a4a1-4ab2-a434-a71269a2dd52/resourceGroups/vpn/providers/Microsoft.Netwo
rk/virtualNetworks/vnet2/subnets/GatewaySubnet"
},
"PublicIpAddress": {
"Id": "/subscriptions/20311952-a4a1-4ab2-a434-a71269a2dd52/resourceGroups/vpn/providers/Microsoft.Netwo
rk/publicIPAddresses/vnet2gwip"
},
"Name": "default",
"Etag": "W/\"af09f17a-88b9-42d7-ab51-c512062430f6\"",
"Id": "/subscriptions/20311952-a4a1-4ab2-a434-a71269a2dd52/resourceGroups/vpn/providers/Microsoft.Network
/virtualNetworkGateways/vnet2gw/ipConfigurations/default"
}
]
GatewayType : Vpn
VpnType : RouteBased
EnableBgp : True
ActiveActive : False
GatewayDefaultSite : null
Sku : {
"Capacity": 2,
"Name": "Standard",
"Tier": "Standard"
}
VpnClientConfiguration : null
BgpSettings : {
"Asn": 65002,
"BgpPeeringAddress": "10.2.254.254",
"PeerWeight": 0
}
Name : vnet3gw
ResourceGroupName : vpn
Location : chinaeast
Id : /subscriptions/20311952-a4a1-4ab2-a434-a71269a2dd52/resourceGroups/vpn/providers/Microsoft.Network/virtualNet
workGateways/vnet3gw
Etag : W/"a558b87f-5898-4bde-88a7-fdd900c762cd"
ResourceGuid : 4b4399e3-1112-4e9e-ad20-280cea25d1ce
ProvisioningState : Succeeded
Tags :
IpConfigurations : [
{
"PrivateIpAllocationMethod": "Dynamic",
"Subnet": {
"Id": "/subscriptions/20311952-a4a1-4ab2-a434-a71269a2dd52/resourceGroups/vpn/providers/Microsoft.Netwo
rk/virtualNetworks/vnet3/subnets/GatewaySubnet"
},
"PublicIpAddress": {
"Id": "/subscriptions/20311952-a4a1-4ab2-a434-a71269a2dd52/resourceGroups/vpn/providers/Microsoft.Netwo
rk/publicIPAddresses/vnet3gwip"
},
"Name": "default",
"Etag": "W/\"a558b87f-5898-4bde-88a7-fdd900c762cd\"",
"Id": "/subscriptions/20311952-a4a1-4ab2-a434-a71269a2dd52/resourceGroups/vpn/providers/Microsoft.Network
/virtualNetworkGateways/vnet3gw/ipConfigurations/default"
}
]
GatewayType : Vpn
VpnType : RouteBased
EnableBgp : True
ActiveActive : False
GatewayDefaultSite : null
Sku : {
"Capacity": 2,
"Name": "Standard",
"Tier": "Standard"
}
VpnClientConfiguration : null
BgpSettings : {
"Asn": 65003,
"BgpPeeringAddress": "10.3.254.254",
"PeerWeight": 0
}
此时3个VPN Gateway的BGP都enable了，同时都设置了AS号码。需要注意的是，Azure内部已经使用了一些BGP的共有、私有的AS号码，在使用是不能冲突：
Public ASNs: 8075, 8076, 12076
Private ASNs: 65515, 65517, 65518, 65519, 65520
d. 配置VPN Gateway的IPSec Connection
配置好后，在三个VPN Gateway间建立Site-to-Site的connection：
for ($i
=
0; $i
-le
2; $i++){
for ($j
=
0; $j
-le
2; $j++){
if($i
-eq
$j)
{
continue
}
else
{
New-AzureRmVirtualNetworkGatewayConnection
-Name
vnet$i-to-vnet$j
-ResourceGroupName vpn
-Location
"China East"
-VirtualNetworkGateway1 $vpngws[$i]
-VirtualNetworkGateway2 $vpngws[$j]
-ConnectionType
Vnet2Vnet
-SharedKey
VPNGW123
-EnableBgp $true
}
}
}
完成后，每个VPN Gateway都和另外两个VPN GW建立了VPN的连接，如图所示：
e. 创建3台测试VM
在每个Vnet中创建一台VM，比如下图可以看到一个NIC部署在VNET1中：
f. 测试联通性
此时可以在VM上进行测试，可以看到VM1、VM2、VM3间通过VPN，都是通的。
在VM1上测试和VM2和VM3的联通性：
[[email protected] ~]# ping 10.2.0.4
PING 10.2.0.4 (10.2.0.4) 56(84) bytes of data.
64 bytes from 10.2.0.4: icmp_seq=1 ttl=62 time=5.65 ms
64 bytes from 10.2.0.4: icmp_seq=2 ttl=62 time=2.33 ms
^C
--- 10.2.0.4 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1001ms
rtt min/avg/max/mdev = 2.338/3.996/5.654/1.658 ms
[[email protected] ~]# ping 10.3.0.4
PING 10.3.0.4 (10.3.0.4) 56(84) bytes of data.
64 bytes from 10.3.0.4: icmp_seq=1 ttl=62 time=7.74 ms
64 bytes from 10.3.0.4: icmp_seq=2 ttl=62 time=6.44 ms
在VM2上测试VM3的联通性：
[[email protected] ~]# ping 10.1.0.4
PING 10.1.0.4 (10.1.0.4) 56(84) bytes of data.
64 bytes from 10.1.0.4: icmp_seq=1 ttl=62 time=5.32 ms
64 bytes from 10.1.0.4: icmp_seq=2 ttl=62 time=2.31 ms
^C
--- 10.1.0.4 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1001ms
rtt min/avg/max/mdev = 2.310/3.816/5.323/1.507 ms
[[email protected] ~]# ping 10.3.0.4
PING 10.3.0.4 (10.3.0.4) 56(84) bytes of data.
64 bytes from 10.3.0.4: icmp_seq=1 ttl=62 time=7.70 ms
g. 测试路由收敛
下面做一个测试，由于1-2 是通过vnet1-vnet2的vpn通起来的，如果断掉这个vpn连接，由于采用了BGP的动态路由，1-2的联通将通过1-3-2的vpn链路连接。
1. 删除VPN连接：
删除后：
VPN的连接如下图：
在VM1上长ping VM2：
[[email protected] ~]# ping 10.2.0.4
PING 10.2.0.4 (10.2.0.4) 56(84) bytes of data.
64 bytes from 10.2.0.4: icmp_seq=1 ttl=62 time=2.52 ms
64 bytes from 10.2.0.4: icmp_seq=2 ttl=62 time=2.36 ms
64 bytes from 10.2.0.4: icmp_seq=3 ttl=62 time=2.62 ms
64 bytes from 10.2.0.4: icmp_seq=4 ttl=62 time=2.60 ms
64 bytes from 10.2.0.4: icmp_seq=5 ttl=62 time=5.78 ms
64 bytes from 10.2.0.4: icmp_seq=6 ttl=62 time=4.11 ms
64 bytes from 10.2.0.4: icmp_seq=7 ttl=62 time=2.15 ms
64 bytes from 10.2.0.4: icmp_seq=8 ttl=62 time=2.43 ms
64 bytes from 10.2.0.4: icmp_seq=9 ttl=62 time=2.62 ms
64 bytes from 10.2.0.4: icmp_seq=10 ttl=62 time=1.93 ms
64 bytes from 10.2.0.4: icmp_seq=11 ttl=62 time=2.16 ms
64 bytes from 10.2.0.4: icmp_seq=12 ttl=62 time=3.71 ms
64 bytes from 10.2.0.4: icmp_seq=26 ttl=61 time=10.9 ms
64 bytes from 10.2.0.4: icmp_seq=27 ttl=61 time=73.2 ms
64 bytes from 10.2.0.4: icmp_seq=28 ttl=61 time=146 ms
64 bytes from 10.2.0.4: icmp_seq=29 ttl=61 time=202 ms
64 bytes from 10.2.0.4: icmp_seq=30 ttl=61 time=26.0 ms
64 bytes from 10.2.0.4: icmp_seq=31 ttl=61 time=11.5 ms
64 bytes from 10.2.0.4: icmp_seq=32 ttl=61 time=59.4 ms
64 bytes from 10.2.0.4: icmp_seq=33 ttl=61 time=44.0 ms
中间出现了14个丢包后，路由收敛，网络继续通了。
h. 结论
从上面的测试中可以看到，Azure的VPN GW支持BGP的动态路由协议。在链路出现故障时，可以通过路由协议实现路由收敛。
2. 与OnPrem的VPN GW连接，通过BGP实现冗余连接
a. 整体的拓扑结构如下：
b. 配置IPSec和BGP
3个VNET连接后，再连接两台Cisco的CSR设备，模拟OnPrem的两个节点。
Cisco的配置请参考Azure VPN的建议配置。
BGP配置如下：
router bgp 65004
bgp log-neighbor-changes
network 10.80.0.0 mask 255.255.255.0
network 10.80.1.0 mask 255.255.255.0
neighbor 10.1.254.254 remote-as 65001
neighbor 10.1.254.254 ebgp-multihop 255
BGP信息：
csr116031aD3v1#sh ip bgp su
BGP router identifier 169.254.0.1, local AS number 65004
BGP table version is 23, main routing table version 23
12 network entries using 2976 bytes of memory
12 path entries using 1440 bytes of memory
5/5 BGP path/bestpath attribute entries using 1240 bytes of memory
4 BGP AS-PATH entries using 144 bytes of memory
0 BGP route-map cache entries using 0 bytes of memory
0 BGP filter-list cache entries using 0 bytes of memory
BGP using 5800 total bytes of memory
BGP activity 12/0 prefixes, 12/0 paths, scan interval 60 secs
Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd
10.1.254.254 4 65001 161 147 23 0 0 02:07:17 10
BGP的路由信息：
csr116031aD3v1#sh ip bgp
BGP table version is 23, local router ID is 169.254.0.1
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,
r RIB-failure, S Stale, m multipath, b backup-path, f RT-Filter,
x best-external, a additional-path, c RIB-compressed,
Origin codes: i - IGP, e - EGP, ? - incomplete
RPKI validation codes: V valid, I invalid, N Not found
Network Next Hop Metric LocPrf Weight Path
*> 10.1.0.0/16 10.1.254.254 0 65001 i
*> 10.2.0.0/16 10.1.254.254 0 65001 65002 i
*> 10.2.254.254/32 10.1.254.254 0 65001 i
*> 10.3.0.0/16 10.1.254.254 0 65001 65003 i
*> 10.3.254.254/32 10.1.254.254 0 65001 i
*> 10.80.0.0/24 0.0.0.0 0 32768 i
r> 10.80.0.22/32 10.1.254.254 0 65001 i
*> 10.80.1.0/24 0.0.0.0 0 32768 i
*> 10.88.0.0/24 10.1.254.254 0 65001 i
*> 10.90.0.0/24 10.1.254.254 0 65001 65002 i
*> 10.90.0.4/32 10.1.254.254 0 65001 65002 i
*> 10.90.1.0/24 10.1.254.254 0 65001 65002 65005 i
在Azure上的配置Local Gateway：
并设置Local Gateway的BGP属性：
$lgw
=
Get-AzureRmLocalNetworkGateway
-ResourceGroupName
vpn
-Name
csrbgp
$lgw.BgpSettings = $vpngws[0].BgpSettings
$lgw.BgpSettings.Asn=65004
$lgw.BgpSettings.BgpPeeringAddress="10.80.0.22"
$lgw.GatewayIpAddress =
"42.159.238.213"
Set-AzureRmLocalNetworkGateway
-LocalNetworkGateway
$lgw
配置VPN的Connection：
New-AzureRmVirtualNetworkGatewayConnection
-Name
tocsrbgp
-ResourceGroupName
vpn
-Location
"China East"
-VirtualNetworkGateway1
$vpngws[0]
-LocalNetworkGateway2
$lgw
-ConnectionType
IPsec
-SharedKey
VPNGW123
-EnableBgp
$true
如此配置两个OnPrem的VPN Gateway。组成上图中的网络。
c. 测试
在VM1上可以ping通Cisco CSR的接口地址
[[email protected] ~]# ping 10.80.1.22
PING 10.80.1.22 (10.80.1.22) 56(84) bytes of data.
64 bytes from 10.80.1.22: icmp_seq=1 ttl=254 time=8.56 ms
64 bytes from 10.80.1.22: icmp_seq=2 ttl=254 time=6.29 ms
[[email protected] ~]# ping 10.90.1.4
PING 10.90.1.4 (10.90.1.4) 56(84) bytes of data.
64 bytes from 10.90.1.4: icmp_seq=1 ttl=253 time=34.6 ms
64 bytes from 10.90.1.4: icmp_seq=2 ttl=253 time=34.6 ms
在两台CSR上也可以互相Ping通：
csr116031aD3v1#ping
Protocol [ip]:
Target IP address: 10.90.1.4
Repeat count [5]: 20
Datagram size [100]:
Timeout in seconds [2]:
Extended commands [n]: y
Ingress ping [n]:
Source address or interface: gi 2
Type of service [0]:
Set DF bit in IP header? [no]:
Validate reply data? [no]:
Data pattern [0x0000ABCD]:
Loose, Strict, Record, Timestamp, Verbose[none]:
Sweep range of sizes [n]:
Type escape sequence to abort.
Sending 20, 100-byte ICMP Echos to 10.90.1.4, timeout is 2 seconds:
Packet sent with a source address of 10.80.1.22
!!!.!!!!!!!!!!!!!.!!
Success rate is 90 percent (18/20), round-trip min/avg/max = 39/40/44 ms
如上面的方法，删除Vnet1到Vnet2间的VPN通路，有部分丢包后，继续ping通。
由于测试采用的是国内一台CSR设备，国外一台CSR设备，网络质量有一些不好，但中间收敛的时间段仍然可以看出来：
csr116031aD3v1#ping
Protocol [ip]:
Target IP address: 10.90.1.4
Repeat count [5]: 200000
Datagram size [100]:
Timeout in seconds [2]:
Extended commands [n]: y
Ingress ping [n]:
Source address or interface: gi 2
Type of service [0]:
Set DF bit in IP header? [no]:
Validate reply data? [no]:
Data pattern [0x0000ABCD]:
Loose, Strict, Record, Timestamp, Verbose[none]:
Sweep range of sizes [n]:
Type escape sequence to abort.
Sending 200000, 100-byte ICMP Echos to 10.90.1.4, timeout is 2 seconds:
Packet sent with a source address of 10.80.1.22
!!!!!!!!!!!!!.!!!!!!!.!!!!!!!!!!.!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!.!!!
!!!!!!!!!!!!!!!!!!!!!.!!!.!!!!!!!!.!!!!!!!!!!.!!!!!!!!!!!!!!.!!!!!!!!!
!!!!......!!!!!!!!!!!!!!!!!!.!!!!.!!!!!!.!!!!!!!!!!!!!!.!!!!!!!!!!!!!!
!!..!!!!!!!!!!!!.!!!!!.!!!!!!!!!!!!.!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
三、结论：
Azure的VPN Gateway目前支持动态路由协议BGP
通过BGP可以实现路由的冗余和HA
OnPrem的VPN Gateway可以通过BGP和VPN与Azure连接，实现冗余的网络结构。