JCNR brings a lot of value by providing seamless connectivity between workloads across locations, public cloud boundaries, and workload form-factor, by providing full router functionality.
Author would like to acknowledge and thank Oleg Berzin and Vinod Nair (Juniper Networks) for their help in putting this solution together in such a short time.
In this day and age of hybrid cloud, it is very common for applications for a given user or organization to span across different datacenter (on-prem or colo) locations, across multiple public cloud providers. Furthermore, they may exist in the form of bare metal, virtual machine or container form-factors. Often apps require seamless connectivity at both Layer 2 and Layer 3 that cuts across cloud boundaries, which is not easy to achieve. Moreover, the typical VPC networking provided by public cloud environments is use-case specific and poses a lot of restrictions as compared to traditional DC networking constructs. This is where JCNR brings a lot of value by providing seamless connectivity between workloads across locations, public cloud boundaries, and workload form-factor, by providing full router functionality.
"Juniper Cloud Native Router (JCNR) is a new breed of Cloud-native Network Functions (CNFs) that have been designed to integrate directly and organically with Kubernetes and deliver a rich set of advanced multi-tenant networking features as well as very-high packet processing and forwarding performance using minimal CPU resources. These characteristics make JCNR applicable not only to the Telco/5G use cases but to a wide variety of enterprise scenarios including hybrid multi-cloud networking. With Equinix Metal and Fabric we can demonstrate that customers can deploy and configure performant hybrid multi-cloud networks using JCNR software on Equinix infrastructure to enable multi-metro, multi-cloud private low latency connectivity for their applications in the matter of minutes."
Oleg Berzin - Senior Distinguished Engineer, Technology and Architecture, Office of the CTO, Equinix.
Equinix Metal is a bare metal infrastructure, which is based on dedicated physical servers, that center on time-to-market, overhead reductions, cloud-like consumption, and integration with other Equinix services in addition to providing high-speed connectivity to public cloud infrastructure using Equinix's high-speed global network infrastructure.
Juniper Cloud-Native Router (JCNR) is a Kubernetes container-based software router that combines the Junos cRPD control and management plane and the vRouter forwarding plane. Telcos are deploying JCNR in their production environments for use-cases such as a cloud-native Cell Site Router (CSR) in the case of 5G Distributed Radio Access Network (D-RAN) and to provide advanced network connectivity and segmentation for 5G Core workloads hosted on public cloud environments.
In addition to the 5G use cases, JCNR is an attractive solution for Enterprises to achieve multi-site, multi-cloud networking for application workloads which this article focuses on.
High-Level Architecture of an Equinix Metal Node with JCNR as Kubernetes CNI¶
The diagram below shows the high-level architecture of JCNR running on top of an Equinix Metal server that is orchestrated on the Equinix Metal infrastructure. Equinix Metal infrastructure along with Equinix's global high-speed backbone provides the required fabric connectivity (L3) between the various JCNR nodes to achieve seamless connectivity between the applications across clouds.
As shown in the diagram below the key JCNR components consists of the following:
Figure 1: JCNR Equinix Architecture
Juniper Containerized Route Processing Daemon or cRPD is the brain of JCNR that runs as a stateful-set on each of the Kubernetes nodes providing advanced networking capabilities for Kubernetes application pods and which talks to the JCNR forwarding plane (vRouter) using GRPC APIs to program configuration data, control-plane state information and to exchange operational state with the vRouter forwarding plane.
CRPD, just like Junos running on a physical Juniper router, provides support for most of the Layer2/Layer3 networking protocols and features such as OSPF, ISIS, BGP, MPLS L2/L3 VPNs, EVPN-VXLAN, Segment-Routing, ACLs, TWAMP, QOS, etc. In addition, there are a variety of options to manage/automate the JCNR configuration using helm-charts, Netconf-SSH, CLI, etc.
JCNR forwarding plane (vRouter) is a high-performance, DPDK-based forwarding plane developed by Juniper Networks that has direct access to the physical and virtual functions (VF/PF) on the fabric interfaces for high-performance traffic forwarding. In addition, pods can be attached to the JCNR through Kubernetes Network Attachment Definition (NAD) construct using vhost (DPDK application-based pod) or veth (kernel mode) interfaces.
JCNR vRouter forwarding plane also provides a native telemetry interface for fetching rich telemetry and metrics information that can be ingested into any standard 3rd party telemetry importer such as Prometheus.
To provide seamless integration of JCNR for Kubernetes workloads, Juniper provides its own JCNR CNI which sits below the Multus meta plugin. The job of the JCNR CNI is to keep a watch on the Kubernetes configuration changes and translate them to equivalent JCNR configuration constructs as and when required.
Note: For in-depth architecture of JCNR please refer to the Junos documentation and other JCNR blog URLs provided at the end of this article in the references section.
This section explains in detail the solution architecture and the use cases that were tested. At a high level, the JCNR Equinix solution aligns with the Equinix Network Edge architecture, a reference architecture from Equinix for connecting multiple locations and clouds. JCNR is deployed in a DIY fashion and plays the role of vGW in this architecture. We will not cover the details of JCNR installation in this post as a detailed installation guide can be found under JCNR Technical documentation for various OS types and environments.
Figure 2: Solution topology and architecture
Use case and test details:
In the Dallas region, 2 JCNR instances are running on 2 separate Equinix metal nodes, and in the Sunnyvale region we have 1 JCNR node. These bare metal servers act as independent Kubernetes clusters for hosting JCNR and other application pods. This is depicted in the diagram as shown above.
Each of these servers is equipped with a dual-port Intel XXV710 25G NIC with the first 25G port connected to Equinix global backbone TOR switches for high-speed inter-POP connectivity. The second port is used for intra-POP connectivity in the Dallas region and in the Sunnyvale region connection towards public cloud AWS-VPC/Azure-VNET.
As indicated in the diagram above, the first 25G port on each of the servers is configured as an untagged port with a single SRIOV-VF allocated to JCNR, whereas multiple SRIOV-VFs are configured on the second 25G port, and are connected to bare metal servers and towards public cloud VPC/VNET interconnects (using different VLANs).
In addition, each metal node has an application pod/CNF attached to JCNR using a Kubernetes native construct known as Network Attachment Definition (NAD).
All the above-mentioned workloads have their JCNR interfaces part of Layer3 VRF vpic-cust1 which is configured on the three JCNR instances. The inter-metro communication between workload endpoints is achieved using the BGP-based L3VPN MPLSoUDP overlay tunnels that are constructed between the three JCNR nodes.
One use case not mentioned in the above diagram is internet connectivity for the above workloads. But that can be achieved very easily by service chaining JCNR with cSRX (which provides Firewall and NAT functions) and breaking out locally on the local metal server for internet access. That's a topic that deserves its own blog!
Note: For the sake of brevity, JCNR Junos configuration and the cli capture from the Linux host and JCNR are provided in the blog repo whose link is provided at the end. Only key verification commands are covered in this document in the next section.
Check for iperf3 TCP throughput between AWS EC2 and Azure VM instances through JCNR-3 over a combination of Azure Express Route and AWS Direct Connect VCs with 500Mbps.
The demonstration of the above use case shows how Juniper Cloud Native Router can be used to provide seamless connectivity between various types of workloads that seamlessly cuts across cloud boundaries. In essence what this means to network operators and enterprise architects is the transformational benefits of a cloud-native networking model while maintaining operational consistency with the existing infrastructure, which means lower opex, faster speed of business, and carrier-grade reliability.
