A Real-world Analysis of ACI Multi-site

Datacenter infrastructure landscape evolving to take various forms and shapes, it is going from a centralized location to co-locations, remote, branch offices, and to the clouds. Clearly, these are sort of influenced by datacenter cost, flexibility, agility and application, and cloud dynamics.

Consequently, the multisite solution from Cisco certainly targeting to address some of these use cases. In this blog let’s do a real-world analysis of ACI multi-site using a reference architecture and its connectivity, software, and hardware requirements.

Real-world Topology Analysis

Cisco ACI multi-site solution is the architecture interconnecting multiple APIC cluster domains with their associated pods. Furthermore, it interconnects separate regions (fabrics) each deployed as either a single pod or multiple pods.  The key software solution element is the multi-site orchestrator/management plane element called MSO. Subsequently, MSO helps with managing the inter-site policies centrally across the on-prem and cloud infrastructure.

From the above diagram, the controls plane function between the DC1 and DC2 is done forming the MP-BGP EVPN session between the SPINE nodes in each site. This allows the exchanging of MAC and IP address information for the endpoints that communicate across the sites. Furthermore, the control plane session can also provide encryption for the DCI link using cloudsec software function.

Additionally, the data plane function between the endpoint connected to DC1 and DC2 are achieved using the site to site VxLAN tunnels.

Key Requirements & Considerations

Latency Requirements: There are three different scenarios to consider when it comes to latency

Software Requirements:  Please note, the MSO software hosting on VMware has the following resource requirements

Topology Requirements

Inter-site spine connectivity: The connectivity between the sites can be done using back-to-back between the two sites or in the case of multiple sites that should be routed through ISN ( Intersite Network nodes). In fact, ISN can be any router or switch model with an L3 function with OSPF, sub-interface, and VLAN tagging support. To demonstrate, the below topology shows the connectivity requirement when routing through the ISN nodes.

MSO orchestrator to the APIC controllers connectivity: The Cisco ACI Multi-Site Orchestrator cluster should sit outside the Cisco ACI fabric – for instance, it could be connected to the APIC using OOB network

Cloud APIC Deployments: The cloud APIC is the software solution, only required to be part of your public cloud infrastructure if you need to extend the network & security policies. Therefore, Ideally, the required policies are globally defined on MSO and then distributed to the Cloud APIC. It accomplishes the task by translating the received policies from the MSO to the cloud-native network policies. For example, it can automate the provisioning of the needed cloud-native resources, such as VPCs, VNET, CSR 1000v deployments, and VGW, security groups, rules, etc.

Summary & Next Steps

In summary, the multi-site solution is ideal if your data center is situated across the cities, regions, or in public clouds. Please note that you can also have a cloud-only deployment with MSO and cloud APIC. Overall, a multisite solution allows optimized stretching of L2/L3 traffic across the hybrid and multi-cloud and also provides centralized management for policies using a software orchestrator called MSO.  For more information about Cisco’s multisite design please visit Cisco multi-site whitepaper and for other Cisco DC solutions please visit Cisco in the categories section. Happy Learning