Learn about Microsoft’s network architecture and products in this article by Florian Klaffenbach, a technology solution professional at Microsoft who is a well-known expert when it comes to hybrid cloud scenarios, cloud connectivity, and cloud environment optimization.
This article will take a closer look at Microsoft’s connectivity options and how you can connect to Microsoft Cloud Services. You will learn about Microsoft’s points of presence for internet peering and products such as Microsoft Azure ExpressRoute and Azure Virtual WAN.
Public interconnect and points of presence
First, you need to know about Microsoft’s points of presence. Microsoft is the second-largest network provider on the globe, and this is also reflected in their PoPs. For public peering, Microsoft currently offers over 160 locations, and for private peering, they are available in more than 50 locations. Public and private peering is all handled via ASN 8075.
The following screenshot shows the entry on http://peeringdb.com for Microsoft AS8075:
Microsofts ExpressRoute’s PoPs are also known, and can be found in the Microsoft documentation. The peering that AS Microsoft is using for ExpressRoute is known as ASN 12076.
Microsoft has a very restricted and selective peering policy, which means that Microsoft does not peer with routeservers and unknown peers. Microsoft also offers only Private Network Interconnects (PNIs) to other network providers. Customers are not allowed to privately peer. Anyway, Microsoft still agrees to publicly peer with customers. A customer only needs to fit into Microsoft’s peering policies, which basically includes the following rules:
- Peering takes place at an internet exchange
- Public peering AS should be used
- Public peering network should have at least a /24 mask
- There should be an entry in http://peeringdb.com
- The NOC should be reachable 24/7
The following diagram shows you a schematic view of the structure behind a Microsoft PoP or Edge site:
Interconnect via internet (HTTPs)
In general, Microsoft platforms and cloud services are available on a public IP only. There is no option, for example, to reach Office 365 via a private network, even if you use ExpressRoute. This also means that Microsoft cloud services communicate with one another on a public IP, even in the Microsoft global backbone.
Let’s look at an example of a big layer-3 routing device, handling all internal and external requests from Microsoft, ensuring that Microsoft service communications stay on the backbone and are reachable for external access.
The following diagram shows a schematic view of this topology:
If a customer wants to restrict the access to their Azure tenant from the internet and only wants to allow certain IPs or IP ranges to have access, then they need to use an Azure Active Directory feature known as conditional access.
Every piece of traffic you send or receive from Microsoft via the internet is secured and at least 256-bit encrypted or more, a military standard of encryption. Microsoft also protects this traffic; you cannot open and inspect those packages sent from or to Microsoft cloud services without triggering a security mechanism that results in package drops. With every opened package, Microsoft suspects a “man-in-the-middle” attack and will prevent corrupted traffic from passing.
Microsoft also secures its endpoints from DDoS attacks by default, which means that all customers using Microsoft IP addresses have this protection.
Most of the Microsoft services are built for the internet, as you can see from the following statement from Microsoft documentation:
There is only one scenario left: to get ExpressRoute to connect to Microsoft 365 services, as shown in the following screenshot (this can be found in the documentation at https://docs.microsoft.com/en-us/office365/enterprise/azure-express…):
Azure VPN
Azure virtual network gateways are core routers within an Azure virtual network. They connect an Azure network to different kinds of interconnect options. Those options are site-to-site VPN, point-to-site VPN, Azure virtual WAN, or ExpressRoute.
Every VNet can have at least one VPN gateway. VPN gateways are available in different service option with different features and available services. With Microsoft VPN or virtual network gateways, you have the following options to connect to your on-premises environment:
- Policy based: IPSec IKEv1, single-site connection with static routing
- Route based: IPSec IKEv2, multisite connection with static routing and BGP
Depending on the devices you connect to on-premises, you can either choose policy or route based.
VPN with network virtual appliances in Azure
In addition to Microsoft’s own offering regarding VPN and virtual gateways, it is also possible to integrate network virtual appliances, such as those available from Palo Alto, Barracuda, CheckPoint, and many other companies, into your virtual-machine environment on Azure. These VMs behave like their physical counterparts in most the cases, and enable more options regarding VPNs. As with Barracuda, you can also use a TINA Tunnel for VPN, a Cisco Miraki SDWAN, and so on.
NVAs are a good extra option if Microsoft has no network offering that fits your cloud strategy.
To find these products, you need to take a look at the Microsoft Azure Marketplace and order it via a third-party solution, as shown in the following screenshot:
You can also set up your own network appliances on Azure as you can with Ubiquiti Unifi, where you can set up a Unifi controller on Azure or AWS to connect your locations and hardware to the cloud. The following screenshot shows a setup on Azure:
Hosting such network controller devices in cloud environments is a very interesting option to reduce the hardware footprint for management on premises, as shown in the following screenshot:
Private network interconnects
When talking about private network interconnects, we’re mainly talking about connecting directly to cloud services without touching any public networks or internet breakouts. This means that you connect through a private network, such as an MPLS, layer-2 networks, or a colocation, to a cloud provider using a connectivity partner, such as Verizon, Equinix, Megaport, or AT&T. The following part will look at Microsoft’s options for private interconnects, and will explain how to use these technologies.
Azure ExpressRoute
When working with ExpressRoute, we are working with a very common technology that is commonly used by Internet Service Providers (ISPs). This technology is calledMultiprotocol Label Switching (MPLS) or ISP IP VPN.
MPLS is a data-transfer technology for widely used networks that routes data from one network to another. The technology works using the shortest path labels and does not use long network addresses. This reduces overhead and long routing tables. MPLS can use the encapsulation of different network protocols and supports nearly every commonly used access technology (for example: T1/E1, frame relay dark fiber).
The routing in these wide-area networks is based on Border Gateway Protocol (BGP) routing. BGP is a standard, and is available with every enterprise layer-3 network gateway device. It has been designed to exchange the routing and availability information of different systems in WANs and other internet connectivity systems. BGP makes routing decisions based on the path, network policies, and rule set configured by a network provider or administrator. BGP then makes the core routing decisions for these networks.
Normally, you see MPLS when connecting a range of offices or data centers to very complex routing or mashed networks among network sites. MPLS also does not terminate Quality of Service (QoS) settings at the gateway; all settings can be transported from one network site to another network site.
With ExpressRoute, Microsoft offers you the option to connect your Azure and Office 365 environment directly to your MPLS network.When you configure your settings, you will be able to configure the different kinds of peering. The two options for peering are as follows:
- Private peering: In general, a customer will use IaaS or other vNet-enabled cloud services. These services can be connected by using a private peering domain. This is an extension of a trusted vNet in Microsoft Azure and an existing tenant infrastructure. Peering can be configured as unidirectional or bidirectional.
- Microsoft peering: Within Microsoft peering, there are services such as Azure Storage, SQL databases, and websites that are offered on public IP addresses. You will be able to connect the Microsoft public peering Azure IP ranges to your DMZ or internal network and connect to all Azure services without having to connect through the internet. You enable bidirectional connectivity among your WAN and Microsoft cloud services.
The following diagram shows the basic schema on the Microsoft site:
What basically happens is that your ISP connects your network to the network of Microsoft. These connections happen at most of the point of presence, meet-me locations, or private network interconnect hubs all over the globe. The following diagram shows how this happens within the Azure data center:
Microsoft offers ExpressRoute in two service levels: Standard SLA and Premium SLA. The premium offering expands the standard offering in the following ways:
- Increased routing table limit from 4,000 routes to 10,000 routes for private peering.
- A higher count of VNets that can be connected to the ExpressRoute circuit (default is 10).
- Global connectivity over the Microsoft core network with ExpressRoute Premium Circuit. This enables you to link a VNet in one geopolitical region with an ExpressRoute circuit in another region. For example, you can link a VNet created in Singapore to an ExpressRoute circuit created in New York.
- Connectivity to Office 365 services and CRM Online.
Azure ExpressRoute Global Reach
Express Route Global Reach provides a link between ExpressRoute circuits to enable a virtual private backbone between a company’s on-premises network, using the Microsoft global backbone. This service makes Microsoft a backbone provider for customers and telecommunication providers. You can even integrate it into an existing ExpressRoute configuration. The following diagram shows a schematic overview of this arrangement:
With ExpressRoute Global Reach, customers get the option to always choose the best and most cost-effective local provider to use to connect to Azure. Customers also gain flexibility when choosing providers, and are able to easily exchange providers or build cheap WANs of their own. ExpressRoute Global Reach is also an option for smaller ISPs and network providers, because they can use ExpressRoute to use Microsoft cables across the Atlantic and Pacific oceans, instead of renting expensive dark fiber cables of their own.
Azure ExpressRoute Direct
ExpressRoute Direct is a service that can connect to Microsoft’s global network at peering locations around the world without needing to connect to other network providers. ExpressRoute Direct provides a redundant 100 Gbps connectivity, which supports active/active connectivity and scalability.The following diagram shows a schematic view of how ExpressRoute Direct works:
ExpressRoute Direct was built for customers who have very high bandwidth needs, as well as network providers who want to use the Microsoft global backbone and sea cables to extend their own network to new regions.
Mixed interconnect with software-defined WANs
Software-defined WAN (SD-WAN), is an acronym for Software-Defined Networking (SDN) and Network Function Virtualization (NFV) in a wide area network (WAN). The implementation of an SD-WAN should make the management and operation of a wide area network easier by separating the networking layer from the control and transport layer. This concept is nearly the same as software-defined networking implementations, where you use virtualization technology to improve data center network management and flexibility and minimize operational overhead.
A key feature of an SD-WAN is that it allows organizations to build higher-performance WANs using less costly and better commercially available internet access or by combining network technologies such as MPLS with other access technologies, such as DSL or fiber internet connections. This enables
businesses to partially or wholly replace expensive private WAN-connection technologies, such as MPLS, or use those technologies for premium workloads, such as data center connections, and use cheaper technologies to connect branches.
The following diagram shows a schematic view of this arrangement:
Azure Virtual WAN
Azure Virtual WAN is a network service from Azure that provides optimized, managed, reportable, and automated branch-to-branch connectivity through a public or a private network with Azure and the Microsoft global backbone. You build an SD-WAN aggregation in the Azure Virtual WAN hub. Virtual WAN lets customers connect and configure SD-WAN devices in branches to communicate with the Azure vWAN Hub. This can be through the manualconfiguration of (for example) devices that do not support automated deployment and configuration, or by using one of Microsoft’s preferred partner appliances and devices that can be managed automatically.
Using preferred partner devices allows you to auto configure the devices, thereby simplifying the management and deployment and enrolling new branches in a few hours, instead of days or weeks.The following diagram shows a schematic view of the vWAN options:
The Azure WAN dashboard provides troubleshooting insights that help you to find failures and issues in your environment. With the dashboard, you can keep even a large-scale connectivity network under control.The following screenshot shows the Azure WAN dashboard:
You now have a deeper look at Microsoft’s network architecture and products that help us to connect our cloud environments. We learned about connecting through the internet and the options for private interconnection. We also learned about the Microsoft global backbone and peering strategy, which should help you to decide the right peering and connectivity options for your future projects. You should now be able to name Microsoft’s interconnect products and limitations, as well as the possible solutions for most of the situations you will face when using a multi-cloud solution.
If you found this article interesting, you can explore Multi-Cloud for Architects as your one-stop guide to work with multiple cloud service providers. Multi-Cloud for Architects will be your go-to guide to find perfect solutions, irrespective of the size of your infrastructure.
