This article is the first in a series on ACX7k Metro Validation. We are covering the following features and their scale.
Layer-2 VPN Technologies provide multipoint services over IP/MPLS networks. The latest addition to this is Ethernet VPN (EVPN) where MAC learning is done via control plane compared to its predecessor Virtual Private LAN Service (VPLS) where MAC got learned via data plane.
RFC-7432 describes BGP MPLS Based Ethernet VPNs with three service types, namely VLAN-Based, VLAN-Bundle, and VLAN-Aware-Bundle. In VLAN-Based MAC-VRF, only one broadcast domain or VLAN is present. In VLAN-Bundle MAC-VRF multiple broadcast domains or VLANs are present, all of them bundled into a single bridge table. In VLAN-Aware Bundle MAC-VRF multiple broadcast domains are present, each having its own bridge table.
JUNOS supports EVPN with MPLS and VxLAN encapsulations. Its E-LAN configuration constructs differ across the platforms. The instance type "mac-vrf" is a unified way to configure EVPN E-LAN across entire Juniper platforms.
ACX7100-32C supports EVPN-MPLS E-LAN functionality with instance-type "mac-vrf". In this test scenario, VLAN-Based and VLAN-Bundle service types of MAC VRF are covered for their scale with 22.2R1 Junos-EVO build. VLAN-Aware Bundle Support is on the roadmap.
6,000 EVPN MAC-VRF Instances were tested on ACX7100-32C with 3,000 VLAN-Based Service Types and 3,000 VLAN-Bundle Service Types. 642,000 MACs across 6,000 MAC-VRF Instances. The same scale is tested on ACX7100-48L as well as on ACX7509.
But the ACX7024 Scale is not covered in this article (same features but different scale are expected on this platform).
Please note that despite sharing the same ACX moniker, the ACX7000 products are different products than ACX500/710/1000/1100/2100/2200/4000/5000/5400/6000. They are powered by different Packet Forwarding Engines (PFE), and support different feature sets and scales.
CEs are simulated using a traffic generator and are connected to PEs using 5x 100G links. PEs are connected to the core using 8x 100G links, bundled in 2x LAGs of four links each.
The ACX7100-32C is the Device Under Test (DUT). The underlay transport used for testing is SR-MPLS with both OSPF and ISIS protocols individually. We also covered LDP as the underlay transport. The CE-bound interface configurations are of SP-Style (see glossary).
For this test, we configured 6,000 MAC-VRF instances:
3,000 with VLAN-Based and the remaining VLAN-Bundle.
VLAN-Based instances have one VLAN and VLAN-Bundle have two VLANs.
43 Local and 43 Remote hosts are learned per VLAN-Based MAC-VRF Instances
64 Local and 64 Remote hosts are learned per VLAN-Bundle MAC-VRF Instances
Total: 642,000 hosts.
Bidirectional traffic in iMIX mode at 99.9% offer-load flows for all the EVPN services. A total of ~1Tbps traffic transits through the DUT.
The first step to verify the EVPN service is to make sure the PE devices are having BGP established with evpn-signalling. As shown below from PE1, BGP is in Established state for the Peer PE2(12.1.1.3) and bgp.evpn.0 table got populated with Type-2 MAC routes.
regress@PE1> show mac-vrf forwarding mac-table summary
Jun 29 05:14:19
Total dynamic and static MAC addresses learned globally : 642000
Configured static MAC addresses learned globally : 0
As shown below, the interface et-0/0/0.1 is bound to MAC-VRF Instance METRO_MAC_VRF_1 with a single vlan (vlan-id 1). The interface is in Forwarding state:
regress@PE1> show evpn instance METRO_MAC_VRF_1 extensive
Instance: METRO_MAC_VRF_1
Route Distinguisher: 12.1.1.1:1
VLAN ID: 1
Per-instance MAC route label: 16
Duplicate MAC detection threshold: 5
Duplicate MAC detection window: 180
MAC database status Local Remote
MAC advertisements: 43 43
MAC+IP advertisements: 0 0
Default gateway MAC advertisements: 0 0
Number of local interfaces: 2 (2 up)
Interface name ESI Mode Status AC-Role
.local..51 00:00:00:00:00:00:00:00:00:00 single-homed Up Root
et-0/0/0.1 00:00:00:00:00:00:00:00:00:00 single-homed Up Root
Number of IRB interfaces: 0 (0 up)
Number of protect interfaces: 0
Number of bridge domains: 1
VLAN Domain-ID Intfs/up IRB-intf Mode MAC-sync v4-SG-sync v6-SG-sync
1 1 1 Extended Enabled Disabled Disabled
Number of neighbors: 1
Address MAC MAC+IP AD IM ES Leaf-label Remote-DCI-Peer
12.1.1.3 43 0 0 1 0
Number of ethernet segments: 0
SMET Forwarding: Disabled
The MAC-VRF Forwarding Table of METRO_MAC_VRF_1 instance captures both local and remote MAC addresses.
regress@PE1> show mac-vrf forwarding mac-table instance METRO_MAC_VRF_1
MAC flags (S - static MAC, D - dynamic MAC, L - locally learned, P - Persistent static, C - Control MAC
SE - statistics enabled, NM - non configured MAC, R - remote PE MAC, O - ovsdb MAC)
Ethernet switching table : 86 entries, 86 learned
Routing instance : METRO_MAC_VRF_1
Vlan MAC MAC Age Logical NH RTR
name address flags interface Index ID
MVRF_VBASED_VLAN_1 00:11:01:00:00:01 D - et-0/0/0.1 0 0
MVRF_VBASED_VLAN_1 00:11:01:00:00:02 D - et-0/0/0.1 0 0
MVRF_VBASED_VLAN_1 00:11:01:00:00:03 D - et-0/0/0.1 0 0
...
MVRF_VBASED_VLAN_1 00:13:01:00:00:01 DC - 202922 202922
MVRF_VBASED_VLAN_1 00:13:01:00:00:02 DC - 202922 202922
MVRF_VBASED_VLAN_1 00:13:01:00:00:03 DC - 202922 202922
...
regress@PE1>
The labels associated with one of the remote MAC learned are displayed with:
regress@PE1> show vlans MVRF_VBUNDLE_VLAN_3001
Routing instance VLAN name Tag Interfaces
METRO_MAC_VRF_3001 MVRF_VBUNDLE_VLAN_3001 NA
.local.202905*
et-0/0/2.1*
et-0/0/3.1001*
regress@PE1> show evpn instance METRO_MAC_VRF_3001 extensive
Instance: METRO_MAC_VRF_3001
Route Distinguisher: 12.1.1.1:3001
Service interface type: VLAN-bundle service interface
Per-instance MAC route label: 6016
Duplicate MAC detection threshold: 5
Duplicate MAC detection window: 180
MAC database status Local Remote
MAC advertisements: 64 64
MAC+IP advertisements: 0 0
Default gateway MAC advertisements: 0 0
Number of local interfaces: 3 (3 up)
Interface name ESI Mode Status AC-Role
.local..2279 00:00:00:00:00:00:00:00:00:00 single-homed Up Root
et-0/0/2.1 00:00:00:00:00:00:00:00:00:00 single-homed Up Root
et-0/0/3.1001 00:00:00:00:00:00:00:00:00:00 single-homed Up Root
Number of IRB interfaces: 0 (0 up)
Number of protect interfaces: 0
Number of bridge domains: 1
VLAN Domain-ID Intfs/up IRB-intf Mode MAC-sync v4-SG-sync v6-SG-sync
None 2 2 Extended Enabled Disabled Disabled
Number of neighbors: 1
Address MAC MAC+IP AD IM ES Leaf-label Remote-DCI-Peer
12.1.1.3 64 0 0 1 0
Number of ethernet segments: 0
SMET Forwarding: Disabled
regress@PE1>
The DUT learns MAC addresses from both the vlans as well as from remote peer over the evpn-signalling.
regress@PE1> show mac-vrf forwarding mac-table instance METRO_MAC_VRF_3001
MAC flags (S - static MAC, D - dynamic MAC, L - locally learned, P - Persistent static, C - Control MAC
SE - statistics enabled, NM - non configured MAC, R - remote PE MAC, O - ovsdb MAC)
Ethernet switching table : 128 entries, 128 learned
Routing instance : METRO_MAC_VRF_3001
Vlan MAC MAC Age Logical NH RTR
name address flags interface Index ID
MVRF_VBUNDLE_VLAN_3001 00:15:01:00:00:01 D - et-0/0/2.1 0 0
MVRF_VBUNDLE_VLAN_3001 00:15:01:00:00:02 D - et-0/0/2.1 0 0
MVRF_VBUNDLE_VLAN_3001 00:15:01:00:00:03 D - et-0/0/2.1 0 0
...
MVRF_VBUNDLE_VLAN_3001 00:19:01:00:00:01 D - et-0/0/3.1001 0 0
MVRF_VBUNDLE_VLAN_3001 00:19:01:00:00:02 D - et-0/0/3.1001 0 0
MVRF_VBUNDLE_VLAN_3001 00:19:01:00:00:03 D - et-0/0/3.1001 0 0
...
MVRF_VBUNDLE_VLAN_3001 00:1b:01:00:00:01 DC - 202905 202905
MVRF_VBUNDLE_VLAN_3001 00:1b:01:00:00:02 DC - 202905 202905
MVRF_VBUNDLE_VLAN_3001 00:1b:01:00:00:03 DC - 202905 202905
...
MVRF_VBUNDLE_VLAN_3001 00:1f:01:00:00:01 DC - 202905 202905
MVRF_VBUNDLE_VLAN_3001 00:1f:01:00:00:02 DC - 202905 202905
MVRF_VBUNDLE_VLAN_3001 00:1f:01:00:00:03 DC - 202905 202905
...
regress@PE1>
The following output captures the number of instances and total MAC learned in the DUT
#! /usr/bin/python
"""
FileName: create_evpn_vrf.py
Version: 1.0
Description: This script will create evpn vrf instances and configure it on the router
Author: Suneesh Babu
"""
import yaml
from glob import glob
from jinja2 import Template
import ipaddress
from jnpr.junos.utils.config import Config
from jnpr.junos import Device
from jnpr.junos.factory import loadyaml
from jnpr.junos.op import *
def iflrange(ifd_name, start_unit, max):
"""
This subroutine yields the specified number of l3 ifls for the ifd
"""
while(max):
iflname = ifd_name + '.' + str(start_unit)
yield iflname
start_unit += 1
max -= 1
def router_operation(config_filename, data, mode):
"""
This sub-routine connects to the router and does the necessary commit operations
"""
router = Device(host=data['host'], user=data['username'], password=data['password'], port=22)
router.open()
cfg = Config(router)
if mode == 'jinja':
cfg.load(template_path=config_filename, template_vars=data, format='text', merge=True)
elif mode == 'setfile':
cfg.load(path=config_filename, format='set')
cfg.pdiff()
cfg.commit()
router.close()
def jinja_template_input(data):
"""
This sub-routine yields the variables required for the jinja template
"""
ifl_unit_list = range(int(data['ifl_start_unit']), int(data['ifl_start_unit']) + int(data['vrf_max']))
vlan_list = range(int(data['vlan_id_start']), int(data['vlan_id_start']) + int(data['vrf_max']))
vrf_id_list = range(int(data['vrf_id']), int(data['vrf_id']) + int(data['vrf_max']))
ifl_name_list = iflrange(data['ifd_name'], data['ifl_start_unit'], int(data['vrf_max']))
return zip(ifl_unit_list, vlan_list), zip(vrf_id_list, list(ifl_name_list), vlan_list)
def main():
"""
To Build the desired number of mac-vrf instances
"""
print("Step-1: Read the Variables from the Params File")
with open(glob('evpn_params.yaml')[0]) as fh:
data = yaml.safe_load(fh.read())
print("Step-2: Build the Data Feed for the Jinja Template Input")
ifl_attributes, evpn_attributes = jinja_template_input(data)
data['ifl_variables'] = ifl_attributes
data['evpn_variables'] = evpn_attributes
print("Step-3: Build the configuration file from Jinja Template")
with open(glob('evpn_jinja.j2')[0]) as t_fh:
t_format = t_fh.read()
evpn_snippet = Template(t_format)
# print (evpn_snippet.render(data))
print("Step-4: Load the config into router and commit it")
router_operation('evpn_jinja.j2', data, 'jinja')
if __name__ == '__main__':
main()
ACX7000 Family platforms (ACX7100-32C, ACX7100-48L, ACX7509) with this tested scale are ready for the Metro deployments. This article demonstrate the MAC VRF scale reachable in architecture based on these routers. The EVPN services capabilities are paramount for the next generation Metro Solution requirements. Next TechPost article will be dedicated to the EVPN VPWS scale validation.
