#打卡不停更# calico BGP RR 跨节点通信原创

whale_life

发布于 2022-9-28 16:11

浏览

0收藏

calico BGP RR Model

BGP Full Mesh 缺陷

#打卡不停更# calico BGP RR 跨节点通信-鸿蒙开发者社区
•由于IBGP水平分割的存在，为了保证所有的BGP路由器都能学习到完整的BGP路由，就必须在AS内实现IBGP全互联，这就导致AS内部需要维护大量的BGP连接，从而影响网络性能，路由反射器（Route Reflector，RR）可以“放宽”水平分割原则，解决该问题。

•为保证IBGP对等体之间的连通性，需要在IBGP对等体之间建立全连接关系。假设在一个AS内部有n台设备，那么建立的IBGP连接数就为n(n-1)/2。当设备数目很多时，设备配置将十分复杂，而且配置后网络资源和CPU资源的消耗都很大。在IBGP对等体间使用路由反射器可以解决以上问题。

BGP RR

#打卡不停更# calico BGP RR 跨节点通信-鸿蒙开发者社区
RR在接收BGP路由时

1.如果该路由学习自非Client IBGP对等体，则反射给自己所有的Client；
2.如果路由学习自Client，则反射给所有非Client IBGP对等体和除了该Client之外的所有Client（华为设备可通过命令关闭RR在Client之间的路由反射行为）；
3.如果路由学习自EBGP对等体，则发送给所有Client和非Client IBGP对等体。
#打卡不停更# calico BGP RR 跨节点通信-鸿蒙开发者社区

eNSP 模拟 BGP RR 模式

在 Full Mesh 的基础上， AR1 和 AR2 建立 Peer，
AR2 和 AR3 建立 Peer，AR1 和 AR3 不建立 Peer 对等体，而是在 AR2 上将 AR1 和 AR3 配置为 RR-client
除此以外，其他的和 Full Mesh 配置全部一致。
#打卡不停更# calico BGP RR 跨节点通信-鸿蒙开发者社区
具体细节配置如下所示：

[AR2]bgp 123
[AR2-bgp]router-id 2.2.2.2
[AR2-bgp]peer 1.1.1.1 as-number 123
[AR2-bgp]peer 1.1.1.1 connect-interface l0
[AR2-bgp]peer 3.3.3.3 as-number 123
[AR2-bgp]peer 3.3.3.3 connect-interface l0
[AR2-bgp]peer 1.1.1.1 reflect-client 
[AR2-bgp]peer 3.3.3.3 reflect-client
[AR2-bgp]dis this
[V200R003C00]
#
bgp 123
 router-id 2.2.2.2
 peer 1.1.1.1 as-number 123 
 peer 1.1.1.1 connect-interface LoopBack0
 peer 3.3.3.3 as-number 123 
 peer 3.3.3.3 connect-interface LoopBack0
 #
 ipv4-family unicast
  undo synchronization
  peer 1.1.1.1 enable
  peer 1.1.1.1 reflect-client
  peer 3.3.3.3 enable
  peer 3.3.3.3 reflect-client
#
return
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
15.
16.
17.
18.
19.
20.
21.
22.
23.
24.
25.
26.

calico BGP RR

安装部署

config-bgp-peering

1.关闭 BGP Full Mesh

确定当前部署环境为 BGP Full Mesh

[root@master ~]# calicoctl node status 
Calico process is running.

IPv4 BGP status
+--------------+-------------------+-------+----------+-------------+
| PEER ADDRESS |     PEER TYPE     | STATE |  SINCE   |    INFO     |
+--------------+-------------------+-------+----------+-------------+
| 192.168.0.81 | node-to-node mesh | up    | 14:40:26 | Established |
| 192.168.0.82 | node-to-node mesh | up    | 14:40:25 | Established |
+--------------+-------------------+-------+----------+-------------+

IPv6 BGP status
No IPv6 peers found.
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.

查看本地是否具有 bgpconfiguration，如果已存在，则导出 yaml 后，配置nodeToNodeMeshEnabled 为 false。
如果没有的话，需要创建一个默认的bgpconfiguration，注意，需要将nodeToNodeMeshEnabled 设置为 false ，关闭 BGP Full Mesh。
BGP-Configuration

[root@master ~]# calicoctl get bgpconfiguration
NAME   LOGSEVERITY   MESHENABLED   ASNUMBER   

cat << EOF | calicoctl create -f -
apiVersion: projectcalico.org/v3
kind: BGPConfiguration
metadata:
  name: default
spec:
  logSeverityScreen: Info
  nodeToNodeMeshEnabled: false
  asNumber: 64512
EOF

[root@master ~]# calicoctl get bgpconfiguration default -o yaml
apiVersion: projectcalico.org/v3
kind: BGPConfiguration
metadata:
  creationTimestamp: "2022-05-24T14:46:43Z"
  name: default
  resourceVersion: "9017"
  uid: efb6991a-32d5-4e85-9aa2-df5978e7683f
spec:
  asNumber: 64512
  logSeverityScreen: Info
  nodeToNodeMeshEnabled: false

[root@master ~]# calicoctl node status 
Calico process is running.

IPv4 BGP status
No IPv4 peers found.

IPv6 BGP status
No IPv6 peers found.


1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
15.
16.
17.
18.
19.
20.
21.
22.
23.
24.
25.
26.
27.
28.
29.
30.
31.
32.
33.
34.
35.
36.
37.

2. 指定 RR 路由反射器

我们指定 node1.whale.com 为 RR 路由反射器

[root@master ~]# kubectl get node -o wide 
NAME               STATUS   ROLES                  AGE   VERSION   INTERNAL-IP    EXTERNAL-IP   OS-IMAGE                KERNEL-VERSION                CONTAINER-RUNTIME
master.whale.com   Ready    control-plane,master   10d   v1.23.5   192.168.0.80   <none>        CentOS Linux 7 (Core)   5.4.193-1.el7.elrepo.x86_64   docker://20.10.16
node1.whale.com    Ready    <none>                 10d   v1.23.5   192.168.0.81   <none>        CentOS Linux 7 (Core)   5.4.193-1.el7.elrepo.x86_64   docker://20.10.16
node2.whale.com    Ready    <none>                 10d   v1.23.5   192.168.0.82   <none>        CentOS Linux 7 (Core)   5.4.193-1.el7.elrepo.x86_64   docker://20.10.16

1.
2.
3.
4.
5.
6.

使用 calicoctl 命令进行改动

[root@master ~]# calicoctl get node node1.whale.com -o yaml --export > node1.yaml
1.

添加到如图所示指定位置内容，指定 node1 节点为 RR 反射器，以此类推，集群节点多的时候，分别指定 1,3,5 等奇数节点的 RR 数量。

metadata:
  labels:
    calico-route-reflector: "calico-route-reflector"
spec:
  bgp:
    routeReflectorClusterID: 224.0.0.1
1.
2.
3.
4.
5.
6.

#打卡不停更# calico BGP RR 跨节点通信-鸿蒙开发者社区
修改完毕以后，calicoctl 执行 apply 即可

[root@master ~]# calicoctl apply -f node1.yaml 
Successfully applied 1 'Node' resource(s)
1.
2.

3.配置 RR-client

配置 RR-client 连接到 RR 路由反射器
peer–RR

calicoctl apply -f - <<EOF
kind: BGPPeer
apiVersion: projectcalico.org/v3
metadata:
  name: peer-to-rrs
spec:
  nodeSelector: "!has(calico-route-reflector)"
  peerSelector: has(calico-route-reflector)
EOF
1.
2.
3.
4.
5.
6.
7.
8.
9.

配置 RR – RR 之间进行连接
RR–RR

calicoctl apply -f - <<EOF
kind: BGPPeer
apiVersion: projectcalico.org/v3
metadata:
  name: rrs-to-rrs
spec:
  nodeSelector: has(calico-route-reflector)
  peerSelector: has(calico-route-reflector)
EOF
1.
2.
3.
4.
5.
6.
7.
8.
9.

查看bgppeer

[root@master ~]# calicoctl get bgppeer
NAME          PEERIP   NODE                           ASN   
peer-to-rrs            !has(calico-route-reflector)   0     
rrs-to-rrs             has(calico-route-reflector)    0     
1.
2.
3.
4.

查看 BGP 状态

[root@master ~]# calicoctl node status
Calico process is running.

IPv4 BGP status
+--------------+---------------+-------+----------+-------------+
| PEER ADDRESS |   PEER TYPE   | STATE |  SINCE   |    INFO     |
+--------------+---------------+-------+----------+-------------+
| 192.168.0.81 | node specific | up    | 15:05:41 | Established |
+--------------+---------------+-------+----------+-------------+

IPv6 BGP status
No IPv6 peers found.

1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.

通过 RR 节点的配置，来进行二次确认

我们找到 calico-node 在 node1 节点的配置文件，查看是否含有 BGP RR 的配置，从而确认集群是否成为 RR 的模式。

[root@master ~]# kubectl -n kube-system exec -it calico-node-bntq7  -- bash

[root@node1 /]# cat /etc/calico/confd/config/bird.cfg
function apply_communities ()
{
}

# Generated by confd
include "bird_aggr.cfg";
include "bird_ipam.cfg";

router id 192.168.0.81;

# Configure synchronization between routing tables and kernel.
protocol kernel {
  learn;             # Learn all alien routes from the kernel
  persist;           # Don't remove routes on bird shutdown
  scan time 2;       # Scan kernel routing table every 2 seconds
  import all;
  export filter calico_kernel_programming; # Default is export none
  graceful restart;  # Turn on graceful restart to reduce potential flaps in
                     # routes when reloading BIRD configuration.  With a full
                     # automatic mesh, there is no way to prevent BGP from
                     # flapping since multiple nodes update their BGP
                     # configuration at the same time, GR is not guaranteed to
                     # work correctly in this scenario.
  merge paths on;    # Allow export multipath routes (ECMP)
}

# Watch interface up/down events.
protocol device {
  debug { states };
  scan time 2;    # Scan interfaces every 2 seconds
}

protocol direct {
  debug { states };
  interface -"cali*", -"kube-ipvs*", "*"; # Exclude cali* and kube-ipvs* but
                                          # include everything else.  In
                                          # IPVS-mode, kube-proxy creates a
                                          # kube-ipvs0 interface. We exclude
                                          # kube-ipvs0 because this interface
                                          # gets an address for every in use
                                          # cluster IP. We use static routes
                                          # for when we legitimately want to
                                          # export cluster IPs.
}


# Template for all BGP clients
template bgp bgp_template {
  debug { states };
  description "Connection to BGP peer";
  local as 64512;
  multihop;
  gateway recursive; # This should be the default, but just in case.
  import all;        # Import all routes, since we don't know what the upstream
                     # topology is and therefore have to trust the ToR/RR.
  export filter calico_export_to_bgp_peers;  # Only want to export routes for workloads.
  add paths on;
  graceful restart;  # See comment in kernel section about graceful restart.
  connect delay time 2;
  connect retry time 5;
  error wait time 5,30;
}

# ------------- Node-to-node mesh -------------
# This node (node1.whale.com) is configured as a route reflector with cluster ID 224.0.0.1;
# ignore node-to-node mesh setting.


# ------------- Global peers -------------
# No global peers configured.


# ------------- Node-specific peers -------------




# For peer /host/node1.whale.com/peer_v4/192.168.0.80
protocol bgp Node_192_168_0_80 from bgp_template {
  neighbor 192.168.0.80 as 64512;
  source address 192.168.0.81;  # The local address we use for the TCP connection
  rr client;
  rr cluster id 224.0.0.1;
}


# For peer /host/node1.whale.com/peer_v4/192.168.0.81
# Skipping ourselves (192.168.0.81)


# For peer /host/node1.whale.com/peer_v4/192.168.0.82
protocol bgp Node_192_168_0_82 from bgp_template {
  neighbor 192.168.0.82 as 64512;
  source address 192.168.0.81;  # The local address we use for the TCP connection
  rr client;
  rr cluster id 224.0.0.1;
}

1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
15.
16.
17.
18.
19.
20.
21.
22.
23.
24.
25.
26.
27.
28.
29.
30.
31.
32.
33.
34.
35.
36.
37.
38.
39.
40.
41.
42.
43.
44.
45.
46.
47.
48.
49.
50.
51.
52.
53.
54.
55.
56.
57.
58.
59.
60.
61.
62.
63.
64.
65.
66.
67.
68.
69.
70.
71.
72.
73.
74.
75.
76.
77.
78.
79.
80.
81.
82.
83.
84.
85.
86.
87.
88.
89.
90.
91.
92.
93.
94.
95.
96.
97.
98.
99.
100.
101.

#打卡不停更# calico BGP RR 跨节点通信-鸿蒙开发者社区

至此，我们的 BGP RR 模式已经可以正常创建，而关于跨节点通信，则和 BGP Full Mesh 类似，我们本章节就暂不演示，有需要可以去 BGP Full Mesh 章节仔细查看即可。

分类

标签

51CTO

51CTO博客

51CTO学堂

#打卡不停更# calico BGP RR 跨节点通信原创

calico BGP RR Model

BGP Full Mesh 缺陷

BGP RR

eNSP 模拟 BGP RR 模式

calico BGP RR

安装部署

1.关闭 BGP Full Mesh

2. 指定 RR 路由反射器

3.配置 RR-client

通过 RR 节点的配置，来进行二次确认

目录

订阅鸿蒙技术特刊，精选内容抢先看

51CTO

51CTO博客

51CTO学堂

#打卡不停更# calico BGP RR 跨节点通信 原创

calico BGP RR Model

BGP Full Mesh 缺陷

BGP RR

eNSP 模拟 BGP RR 模式

calico BGP RR

安装部署

1.关闭 BGP Full Mesh

2. 指定 RR 路由反射器

3.配置 RR-client

通过 RR 节点的配置，来进行二次确认

目录

订阅鸿蒙技术特刊，精选内容抢先看

#打卡不停更# calico BGP RR 跨节点通信原创