Service providers started adopting the EVPN technology as a replacement for their L2 technology requirements like VPLS, L2VPN. There is a lot of benefits of EVPN as a technology which is briefly covered in the below section. At the same time service providers also look for the migration of the legacy L2 technologies (which has with a lot of limitations) to the EVPN technology.
This solution provides a lot of flexibility to the service providers in doing a staged migration and the best part is the coexistence of the legacy technologies and EVPN at the same time as described in Figure 1. So, PEs in different segments can run VPLS and EVPN for the same VPN routing instance. Also, the migration supports all the different scenarios like single-homed / multi-homed PEs.
EVPN is a next-generation technology helping the service providers to overcome the limitations of the legacy L2 business services. It's a more efficient technology with better control plane, the data plane learning and programming capabilities.
EVPN over MPLS is considered the main enabler of the Carrier Ethernet services in the Metro Area Networks (MAN) and Mobile BackHaul (MBH) networks by many service providers and is the integral attribute of the Cloud Metro Architecture. It covers all possible overlay service topology: point-to-point, multipoint-to-multipoint and point-to-multipoint, seamlessly. The table below shows the mapping of Layer 2 service definitions with different industry stands and technology including EVPN.
Let's pick a VPLS PE router with multiple instances. The idea of "seamless migration" represents the capability of migrating some instances to EVPN technology while maintaining the VPLS operation on the other instances. The instance migrated to EVPN can also operate with a VPLS instance till the other end VPLS PE is also migrated to EVPN.
Here are some best practices for migration.
In a scaled environment, pick the VPLS PE instances with lower MAC count and least traffic. Do the migration (procedure in details given below) and wait for all the MAC addresses to be learnt and traffic is forwarded successfully.
When all the instances are migrated, the PE is completely migrated to EVPN: it is time to clean-up old configuration. For the BGP-VPLS clean-up, a BGP flap is expected because of signalling and negotiation of BGP capabilities.
Migrate all multihomed PEs to EVPN and do NOT leave some as VPLS-only and migrate others (as VPLS and EVPN use different algorithms to decide Designated forwarder). Multihomed PEs must be in sync regarding the forwarding decision of BUM traffic received from the core towards access.
All four PEs in the above topology are BGP-VPLS PE and have the VPLS instance setup and then relevant configurations are made in each one of these PEs. These PEs discover each other in the network and establishes the VPLS sessions. The configuration and operational status on two of the PE1 and PE4 are shown below, the rest is removed for sake of brevity.
All the four PEs in the topology in Figure 2 are BGP-VPLS Pes. We will perform the migration for one of them, from VPLS to EVPN. Once completed, we will have VPLS PE on one side and EVPN PE on the other.
Migration of PE1: We add the protocol evpn details under the routing-instance and we add the family evpn signalling:
regress@rtme-mx-02# show routing-instances vpls-bgp
instance-type evpn;
protocols {
evpn { <<< Add this part to the routing instance and keep the VPLS config
interface et-0/1/2.1;
}
vpls {
site site1 {
site-identifier 2;
}
no-tunnel-services;
}
}
vlan-id 1234;
interface et-0/1/2.1;
route-distinguisher 192.168.0.2:1;
vrf-target target:1:1;
regress@rtme-mx-02# show protocols bgp group iBGP
type internal;
local-address 192.168.0.2;
family inet {
unicast;
}
family inet-vpn {
unicast;
}
family l2vpn {
signaling;
}
family evpn { <<< Add this to signal the BGP with evpn family to all the neighbours.
<<< You can also specifically add this under the desired neighbour for signalling.
signaling;
}
neighbor 192.168.0.22;
neighbor 192.168.0.62;
neighbor 192.168.0.61;
PE1 still maintains the VPLS status with the other PE, however, the forwarding is now happening using the EVPN technology. We can see the new EVPN tables are created, and the EVPN routes are exchanged between the PEs.
regress@rtme-mx-02# run show vpls connections
Layer-2 VPN connections:
Instance: vpls-bgp
Edge protection: Not-Primary
Local site: site1 (2)
connection-site Type St Time last up # Up trans
22 rmt Up Feb 13 07:33:32 2022 1
Remote PE: 192.168.0.22, Negotiated control-word: No
Incoming label: 117, Outgoing label: 87
Local interface: lsi.1049093, Status: Up, Encapsulation: VPLS
Description: Intf - vpls vpls-bgp local site 2 remote site 22
Flow Label Transmit: No, Flow Label Receive: No
61 rmt Up Feb 13 07:33:32 2022 1
Remote PE: 192.168.0.61, Negotiated control-word: No
Incoming label: 124, Outgoing label: 79
Local interface: lsi.1049094, Status: Up, Encapsulation: VPLS
Description: Intf - vpls vpls-bgp local site 2 remote site 61
Flow Label Transmit: No, Flow Label Receive: No
62 rmt Up Feb 13 07:33:32 2022 1
Remote PE: 192.168.0.62, Negotiated control-word: No
Incoming label: 125, Outgoing label: 67
Local interface: lsi.1049095, Status: Up, Encapsulation: VPLS
Description: Intf - vpls vpls-bgp local site 2 remote site 62
Flow Label Transmit: No, Flow Label Receive: No
regress@rtme-mx-02# run show bgp summary
Threading mode: BGP I/O
Default eBGP mode: advertise - accept, receive - accept
Groups: 2 Peers: 3 Down peers: 0
Table Tot Paths Act Paths Suppressed History Damp State Pending
inet.0
0 0 0 0 0 0
bgp.l3vpn.0
0 0 0 0 0 0
bgp.evpn.0
0 0 0 0 0 0
lsdist.0
0 0 0 0 0 0
bgp.l2vpn.0
9 9 0 0 0 0
Peer AS InPkt OutPkt OutQ Flaps Last Up/Dwn State|#Active/Received/Accepted/Damped...
192.168.0.22 65000 176 178 0 0 1:14:49 Establ
inet.0: 0/0/0/0
bgp.l3vpn.0: 0/0/0/0
bgp.l2vpn.0: 3/3/3/0
vpls-bgp.l2vpn.0: 3/3/3/0
192.168.0.61 65000 176 177 0 0 1:14:49 Establ
inet.0: 0/0/0/0
bgp.l3vpn.0: 0/0/0/0
bgp.l2vpn.0: 3/3/3/0
vpls-bgp.l2vpn.0: 3/3/3/0
192.168.0.62 65000 112 109 0 2 44:49 Establ
inet.0: 0/0/0/0
bgp.l3vpn.0: 0/0/0/0
bgp.l2vpn.0: 3/3/3/0
vpls-bgp.l2vpn.0: 3/3/3/0
regress@rtme-mx-02# run show evpn instance
Intfs IRB intfs MH MAC addresses
Instance Total Up Total Up Nbrs ESIs Local Remote
__default_evpn__ 0
vpls-bgp 2 2 0 0 0 0 0 0
[edit]
regress@rtme-mx-02# run show evpn instance extensive
Instance: __default_evpn__
Route Distinguisher: 192.168.0.2:0
Number of bridge domains: 0
Number of neighbors: 0
Instance: vpls-bgp
Route Distinguisher: 192.168.0.2:1
VLAN ID: 1234
Per-instance MAC route label: 102
Duplicate MAC detection threshold: 5
Duplicate MAC detection window: 180
MAC database status Local Remote
MAC advertisements: 0 0
MAC+IP advertisements: 0 0
Default gateway MAC advertisements: 0 0
Number of local interfaces: 2 (2 up)
Interface name ESI Mode Status AC-Role
.local..8 00:00:00:00:00:00:00:00:00:00 single-homed Up Root
et-0/1/2.1 00:00:00:00:00:00:00:00:00:00 single-homed Up Root
Number of IRB interfaces: 0 (0 up)
Number of protect interfaces: 0
Number of bridge domains: 1
VLAN Domain-ID Intfs/up IRB-intf Mode MAC-sync IM-label MAC-label v4-SG-sync IM-core-NH v6-SG-sync IM-core-NH Trans-ID
1234 1 1 Extended Enabled 128 Disabled Disabled
Number of neighbors: 0
Number of ethernet segments: 0
SMET Forwarding: Disabled
Migration: BGP-VPLS
regress@rtme-mx-02# run show route table bgp.evpn.0
bgp.evpn.0: 1 destinations, 1 routes (1 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both
3:192.168.0.2:1::1234::192.168.0.2/248 IM
*[EVPN/170] 00:10:27
Indirect
PE2, PE3, and PE4 continue to be VPLS PEs, so no change in configuration.
Below is the operational status of PE4, just as a reference that this PE still has the business service up using VPLS on its end while PE1 is now EVPN.
regress@rtme-mx-62-1# run show vpls connections
Layer-2 VPN connections:
Instance: vpls-bgp
Edge protection: Not-Primary
Local site: site1 (62)
connection-site Type St Time last up # Up trans
2 rmt Up Feb 13 07:33:32 2022 1
Remote PE: 192.168.0.2, Negotiated control-word: No
Incoming label: 67, Outgoing label: 125
Local interface: lsi.1048839, Status: Up, Encapsulation: VPLS
Description: Intf - vpls vpls-bgp local site 62 remote site 2
Flow Label Transmit: No, Flow Label Receive: No
22 rmt Up Feb 13 07:33:14 2022 1
Remote PE: 192.168.0.22, Negotiated control-word: No
Incoming label: 79, Outgoing label: 99
Local interface: lsi.1048837, Status: Up, Encapsulation: VPLS
Description: Intf - vpls vpls-bgp local site 62 remote site 22
Flow Label Transmit: No, Flow Label Receive: No
61 rmt Up Feb 13 07:33:14 2022 1
Remote PE: 192.168.0.61, Negotiated control-word: No
Incoming label: 62, Outgoing label: 75
Local interface: lsi.1048838, Status: Up, Encapsulation: VPLS
Description: Intf - vpls vpls-bgp local site 62 remote site 61
Flow Label Transmit: No, Flow Label Receive: No
PE3: BGP-VPLS Instance to EVPN Instance Migration¶
The below section shows the migration of PE3 (VPLS PE) to EVPN PE and checks how it learns the routes from PE1 which is already moved to EVPN technology.
regress@rtme-mx-22# show routing-instances vpls-bgp
instance-type evpn;
protocols {
evpn {
interface et-0/0/2.1;
}
vpls {
site site1 {
site-identifier 22;
}
no-tunnel-services;
}
}
vlan-id 1234;
interface et-0/0/2.1;
route-distinguisher 192.168.0.22:1;
vrf-target target:1:1;
[edit]
regress@rtme-mx-22# show protocols bgp group iBGP
type internal;
local-address 192.168.0.22;
family inet {
unicast;
}
family inet-vpn {
unicast;
}
family l2vpn {
signaling;
}
family evpn {
signaling;
}
neighbor 192.168.0.2;
neighbor 192.168.0.61;
neighbor 192.168.0.62;
PE3 has established an EVPN PE neighborship with PE1 and has learnt the EVPN routes. At the same time, the other two PEs which are still VPLS PEs maintain the VPLS control plane and forwarding state.
All four PEs in the topology in Figure 2 are signalled with LDP-VPLS. We will migrate PEs from VPLS to EVPN one by one. After each migration, we can see how on one end we have VPLS PE and on the other end, we have the EVPN PE.
Here are the configuration changes that need to be done during the migration
For the intended routing instance, change the instance-type from "vpls" to "evpn".
Keep the existing VPLS configuration for now.
Add protocol evpn and the relevant evpn configurations to the routing instance
Add route-distinguisher and route-target to the routing-instance
Under the protocol bgp, add the family evpn signalling
In this section, we are going to migrate the VPLS PE to an EVPN. Only PE1 is migrating to EVPN and the rest will continue to operate as VPLS PEs. After the migration is done the evpn instance shows details about which parameters are programmed including the ESI and the migration model (BGP-VPLS / LDP-VPLS). The highlighted below configurations are the ones that are required for the seamless migration to work. PE1 does move the mac entries from the VPLS database to the EVPN database. However, since there is no other PE in the network that has migrated this PE does not have any EVPN routes learnt.
regress@rtme-mx-22> show vpls connections
Instance: vpls
VPLS-id: 2
Neighbor Type St Time last up # Up trans
192.168.0.2(vpls-id 2) rmt Up Jul 12 01:48:49 2022 1
Remote PE: 192.168.0.2, Negotiated control-word: No
Incoming label: 28, Outgoing label: 103
Negotiated PW status TLV: No
Local interface: vt-0/0/0.1048584, Status: Up, Encapsulation: ETHERNET
Description: Intf - vpls vpls neighbor 192.168.0.2 vpls-id 2
Flow Label Transmit: No, Flow Label Receive: No
192.168.0.61(vpls-id 2) rmt Up Jul 12 01:48:49 2022 1
Remote PE: 192.168.0.61, Negotiated control-word: No
Incoming label: 29, Outgoing label: 41
Negotiated PW status TLV: No
Local interface: vt-0/0/0.1048586, Status: Up, Encapsulation: ETHERNET
Description: Intf - vpls vpls neighbor 192.168.0.61 vpls-id 2
Flow Label Transmit: No, Flow Label Receive: No
192.168.0.62(vpls-id 2) rmt Up Jul 12 01:48:49 2022 1
Remote PE: 192.168.0.62, Negotiated control-word: No
Incoming label: 30, Outgoing label: 16030
Negotiated PW status TLV: No
Local interface: vt-0/0/0.1048585, Status: Up, Encapsulation: ETHERNET
Description: Intf - vpls vpls neighbor 192.168.0.62 vpls-id 2
Flow Label Transmit: No, Flow Label Receive: No
regress@rtme-mx-22> show vpls mac-table
MAC flags (S -static MAC, D -dynamic MAC, L -locally learned, C -Control MAC
O -OVSDB MAC, SE -Statistics enabled, NM -Non configured MAC, R -Remote PE MAC, P -Pinned MAC)
Routing instance : vpls
Bridging domain : __vpls__, VLAN : NA
MAC MAC Logical NH MAC active
address flags interface Index property source
00:cc:cc:cc:00:00 D vt-0/0/0.1048584
Screenshot of the traffic generator console while PE1 migrates showing a slight disruption because of BGP signalling.
In this section, we are going to migrate the VPLS PE (PE3) to an EVPN PE. As seen in the section above PE1 has already migrated to EVPN, so when PE3 migrates the rest will continue to operate as VPLS PEs. After completion of PE1 & PE3 migration, these will continue to be EVPN PE and the other two PEs (PE2 & PE4) will continue to operate as VPLS PE.
regress@rtme-mx-02# run show bgp summary
Threading mode: BGP I/O
Default eBGP mode: advertise - accept, receive - accept
Groups: 1 Peers: 3 Down peers: 0
Table Tot Paths Act Paths Suppressed History Damp State Pending
inet.0
0 0 0 0 0 0
bgp.l3vpn.0
0 0 0 0 0 0
bgp.evpn.0
1 1 0 0 0 0
Peer AS InPkt OutPkt OutQ Flaps Last Up/Dwn State|#Active/Received/Accepted/Damped...
192.168.0.22 65000 11 11 0 1 2:23 Establ
inet.0: 0/0/0/0
bgp.l3vpn.0: 0/0/0/0
bgp.evpn.0: 1/1/1/0
vpls.evpn.0: 1/1/1/0
__default_evpn__.evpn.0: 0/0/0/0
192.168.0.61 65000 60 58 0 0 25:21 Establ
inet.0: 0/0/0/0
bgp.l3vpn.0: 0/0/0/0
192.168.0.62 65000 60 59 0 0 25:21 Establ
inet.0: 0/0/0/0
bgp.l3vpn.0: 0/0/0/0
regress@rtme-mx-02# run show evpn instance vpls extensive
Instance: vpls
Route Distinguisher: 192.168.0.2:100
VLAN ID: 1
Per-instance MAC route label: 101
Duplicate MAC detection threshold: 5
Duplicate MAC detection window: 180
MAC database status Local Remote
MAC advertisements: 1 0
MAC+IP advertisements: 0 0
Default gateway MAC advertisements: 0 0
Number of local interfaces: 2 (2 up)
Interface name ESI Mode Status AC-Role
.local..14 00:00:00:00:00:00:00:00:00:00 single-homed Up Root
et-0/1/2.1 00:00:00:00:00:00:00:00:00:00 single-homed Up Root
Number of IRB interfaces: 0 (0 up)
Number of protect interfaces: 0
Number of bridge domains: 1
VLAN Domain-ID Intfs/up IRB-intf Mode MAC-sync v4-SG-sync v6-SG-sync
1 1 1 Extended Enabled Disabled Disabled
Number of neighbors: 1
Address MAC MAC+IP AD IM ES Leaf-label Remote-DCI-Peer
192.168.0.22 0 0 0 1 0
Number of ethernet segments: 0
SMET Forwarding: Disabled
Migration: LDP-VPLS
regress@rtme-mx-02# run show evpn database
Instance: vpls
VLAN DomainId MAC address Active source Timestamp IP address
1 00:cc:cc:cc:00:00 et-0/1/2.1 Jul 12 02:33:27
regress@rtme-mx-02-1# run show evpn mac-table
MAC flags (S -static MAC, D -dynamic MAC, L -locally learned, C -Control MAC
O -OVSDB MAC, SE -Statistics enabled, NM -Non configured MAC, R -Remote PE MAC, P -Pinned MAC)
Routing instance : vpls
Bridging domain : __vpls__, VLAN : 1
MAC MAC Logical NH MAC active
address flags interface Index property source
00:cc:cc:cc:00:00 D et-0/1/2.1
regress@rtme-mx-02# run show route table vpls.evpn.0
vpls.evpn.0: 3 destinations, 3 routes (3 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both
2:192.168.0.2:100::1::00:cc:cc:cc:00:00/304 MAC/IP
*[EVPN/170] 00:26:03
Indirect
3:192.168.0.2:100::1::192.168.0.2/248 IM
*[EVPN/170] 00:26:01
Indirect
3:192.168.0.22:100::1::192.168.0.22/248 IM
*[BGP/170] 00:03:03, localpref 100, from 192.168.0.22
AS path: I, validation-state: unverified
> to 11.0.0.24 via et-0/0/2.0, Push 313
regress@rtme-mx-02-1# run show route table bgp.evpn.0
bgp.evpn.0: 3 destinations, 3 routes (3 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both
2:192.168.0.2:100::1::00:cc:cc:cc:00:00/304 MAC/IP
*[EVPN/170] 00:26:08
Indirect
3:192.168.0.2:100::1::192.168.0.2/248 IM
*[EVPN/170] 00:26:06
Indirect
3:192.168.0.22:100::1::192.168.0.22/248 IM
*[BGP/170] 00:03:08, localpref 100, from 192.168.0.22
AS path: I, validation-state: unverified
> to 11.0.0.24 via et-0/0/2.0, Push 313
regress@rtme-mx-22# run show bgp summary
Threading mode: BGP I/O
Default eBGP mode: advertise - accept, receive - accept
Groups: 1 Peers: 3 Down peers: 0
Table Tot Paths Act Paths Suppressed History Damp State Pending
inet.0
0 0 0 0 0 0
bgp.l3vpn.0
0 0 0 0 0 0
lsdist.0
0 0 0 0 0 0
bgp.evpn.0
2 2 0 0 0 0
Peer AS InPkt OutPkt OutQ Flaps Last Up/Dwn State|#Active/Received/Accepted/Damped...
192.168.0.2 65000 7 5 0 0 3 Establ
inet.0: 0/0/0/0
bgp.l3vpn.0: 0/0/0/0
bgp.evpn.0: 2/2/2/0
vpls.evpn.0: 2/2/2/0
__default_evpn__.evpn.0: 0/0/0/0
192.168.0.61 65000 4 3 0 0 3 Establ
inet.0: 0/0/0/0
bgp.l3vpn.0: 0/0/0/0
192.168.0.62 65000 4 3 0 0 3 Establ
inet.0: 0/0/0/0
bgp.l3vpn.0: 0/0/0/0
regress@rtme-mx-22# run show evpn instance vpls extensive
Instance: vpls
Route Distinguisher: 192.168.0.22:100
VLAN ID: 1
Per-instance MAC route label: 36
Duplicate MAC detection threshold: 5
Duplicate MAC detection window: 180
MAC database status Local Remote
MAC advertisements: 0 1
MAC+IP advertisements: 0 0
Default gateway MAC advertisements: 0 0
Number of local interfaces: 2 (2 up)
Interface name ESI Mode Status AC-Role
.local..9 00:00:00:00:00:00:00:00:00:00 single-homed Up Root
et-0/0/2.1 00:00:00:00:00:00:00:00:00:00 single-homed Up Root
Number of IRB interfaces: 0 (0 up)
Number of protect interfaces: 0
Number of bridge domains: 1
VLAN Domain-ID Intfs/up IRB-intf Mode MAC-sync v4-SG-sync v6-SG-sync
1 1 1 Extended Enabled Disabled Disabled
Number of neighbors: 1
Address MAC MAC+IP AD IM ES Leaf-label Remote-DCI-Peer
192.168.0.2 1 0 0 1 0
Number of ethernet segments: 0
SMET Forwarding: Disabled
Migration: LDP-VPLS
regress@rtme-mx-22# run show evpn database
Instance: vpls
VLAN DomainId MAC address Active source Timestamp IP address
1 00:cc:cc:cc:00:00 192.168.0.2 Jul 12 02:33:25
regress@rtme-mx-22# run show evpn mac-table
MAC flags (S -static MAC, D -dynamic MAC, L -locally learned, C -Control MAC
O -OVSDB MAC, SE -Statistics enabled, NM -Non configured MAC, R -Remote PE MAC, P -Pinned MAC)
Routing instance : vpls
Bridging domain : __vpls__, VLAN : 1
MAC MAC Logical NH MAC active
address flags interface Index property source
00:cc:cc:cc:00:00 DC 1048578 192.168.0.2
regress@rtme-mx-22# run show route table vpls.evpn.0
vpls.evpn.0: 3 destinations, 3 routes (3 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both
2:192.168.0.2:100::1::00:cc:cc:cc:00:00/304 MAC/IP
*[BGP/170] 00:01:32, localpref 100, from 192.168.0.2
AS path: I, validation-state: unverified
> to 11.0.0.23 via et-1/0/5.0, Push 305
3:192.168.0.2:100::1::192.168.0.2/248 IM
*[BGP/170] 00:01:32, localpref 100, from 192.168.0.2
AS path: I, validation-state: unverified
> to 11.0.0.23 via et-1/0/5.0, Push 305
3:192.168.0.22:100::1::192.168.0.22/248 IM
*[EVPN/170] 00:01:33
Indirect
regress@rtme-mx-22# run show route table bgp.evpn.0
bgp.evpn.0: 3 destinations, 3 routes (3 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both
2:192.168.0.2:100::1::00:cc:cc:cc:00:00/304 MAC/IP
*[BGP/170] 00:01:36, localpref 100, from 192.168.0.2
AS path: I, validation-state: unverified
> to 11.0.0.23 via et-1/0/5.0, Push 305
3:192.168.0.2:100::1::192.168.0.2/248 IM
*[BGP/170] 00:01:36, localpref 100, from 192.168.0.2
AS path: I, validation-state: unverified
> to 11.0.0.23 via et-1/0/5.0, Push 305
3:192.168.0.22:100::1::192.168.0.22/248 IM
*[EVPN/170] 00:01:37
Indirect
Screenshot of traffic generator while PE1 migrates showing a slight disruption because of BGP signalling.
