Microsoft ExpressRoute Overview

PacketFabric’s Hosted cloud connection follows Microsoft’s CloudExchange Co-location connectivity model.

Microsoft ExpressRoute circuits are private, dedicated connections between your on-premises network and your Azure cloud environment. Because they bypass the public internet, these circuits are much more efficient, secure, and often more cost-effective than traditional connections. For more information about its benefits, see Microsoft’s ExpressRoute service overview.

A single ExpressRoute circuit comprises the following:

  • Primary connection
    • Primary peering link to Azure virtual network
    • Primary peering link to Microsoft 365
  • Secondary connection
    • Secondary peering link to Azure virtual network
    • Secondary peering link to Microsoft 365

full ExpressRoute diagram

NOTE: This does not mean you have to use both connections and all four peerings. For example, you can provision only the primary connection, and then only peer with your Azure VNet.

However, if you do not provision the secondary connection, this could affect how you are covered under Microsoft’s SLA.

For more information, see High Availability and Redundancy in ExpressRoute Connections.

Locations

You can connect to the PacketFabric network at any of our locations.

For a list of on-ramps, see https://packetfabric.com/locations/cloud-on-ramps.

Pricing

PacketFabric prices

See the prices that are posted on our website: https://packetfabric.com/pricing

ExpressRoute prices

See Microsoft’s Azure ExpressRoute pricing page and Pricing Calculator.

Bandwidth

For Hosted connections, you can select the following bandwidths for your circuit: 50 Mbps, 100 Mbps, 200 Mbps, 500 Mbps, 1 Gbps, 2 Gbps, 5 Gbps, and 10 Gbps

Dedicated connections are 10 Gbps or 100 Gbps.

Note the following:

  • Bandwidth is full duplex, so you get full bandwith going both ways across a connection.
  • The bandwidth you select for the circuit is replicated between the primary and secondary connection. So if you create a 100 Mbps circuit, you can create a 100 Mbps primary connection and a 100 Mbps secondary connection.
  • The bandwidth is shared among the peerings in a connection. To use the example from above again, if you create a 100 Mbps circuit, you can create two 100 Mbps connections (primary and secondary). If you implement both peerings (Azure private and Microsoft) within those connections, then each peering will have 50 Mbps.

For more information, see Microsoft - If I pay for an ExpressRoute circuit of a given bandwidth, do I have the ability to use more than my procured bandwidth?.

Peering

As illustrated above, each ExpressRoute connection allows two types of peerings: Azure private and Microsoft. Both are BGP sessions. Both peering types support MD5 hashes.

For more information, see Microsoft - ExpressRoute circuits and peering.

Azure private peering

This is a bi-directional connection between your core on-premises network and one or more Azure virtual networks. It uses a private peering domain to essentially extend your network directly into Azure, creating secure, high-speed access to Azure cloud services and VMs.

Within each virtual network, you can have multiple subnets. You can also use peering to link multiple virtual networks to each other. Furthermore, you can connect to an entire VNet fabric via one ExpressRoute circuit (limits apply).

ExpressRoute Azure private peering diagram

NOTE: For simplicity, the above diagram does not show the PacketFabric ports that complete the ExpressRoute circuit.

Microsoft peering

Microsoft peering is a bi-directional connection between your on-premises network and select Microsoft 365 services.

The Microsoft peering option is quite a bit more complicated than Azure private peering. Before you can initiate Microsoft 365/Office 365 traffic through an ExpressRoute circuit, you must first request authorization from Microsoft: ExpressRoute for Office 365 Request Form

Note the following rules:

  • All traffic to Microsoft 365 must originate from a valid public IPv4 address. Do not advertise the same public IP route to the public internet.

  • All public ASNs and IP addresses must be registered in one of Microsoft’s pre-approved registries. If they are not registered, you will need to open a support case to have them manually validated.

  • If you want to use a private ASN, you will need to open a support case to have it manually validated. Using a private ASN will also limit your ability to optimize route paths. For more information, see Microsoft - Suboptimal routing from Microsoft to customer.

  • When an ExpressRoute peering is present, the Microsoft 365 front-end servers will favor circuit routes over routing via the public internet. This could lead to route asymmetry if your network is prioritizing internet routes. For more information, see Microsoft - Ensuring route symmetry.

  • You must use SNAT for incoming and outgoing traffic. For more information, see Microsoft - ExpressRoute NAT requirements.

  • Not all Microsoft 365 services are reachable through ExpressRoute, and the ones that are reachable still have some public internet requirements. For more information, see Microsoft - What Office 365 services are included? and Microsoft - Office 365 URLs and IP address ranges.

  • Instead of setting up a gateway as you do with Azure private peering, you must configure a route filter. This enables Microsoft route advertisements to your network.

For more information, see Microsoft - ExpressRoute for Office 365.

Azure ExpressRoute Microsoft peering diagram

NOTE: For simplicity, the above diagram does not show the PacketFabric ports that complete the ExpressRoute circuit.

Additional technical notes

Virtual circuits

  • Row
    • Field
    • Description
  • Row
    • Connection Speeds
    • 50 Mbps, 100 Mbps, 200 Mbps, 500 Mbps, 1 Gbps, 2 Gbps, 5 Gbps, 10 Gbps
  • Row
    • Burst Speeds
    • You can burst up to 2x your purchased bandwidth at no cost. This does not apply to traffic flowing through an ExpressRoute gateway. Source
  • Row
    • MTU
    • 1500 bytes
  • Row
    • TCP MSS
    • Does not need to be specified
  • Row
    • BFD
    • Supported

Peering configurations

  • Row
    • Requirement
    • Azure private peering
    • Microsoft peering
  • Row
    • Max prefixes
    • 4000 (or 10,000 with ExpressRoute premium)
    • 200
  • Row
    • Supported IP protocols
    • IPv4
    • IPv4 and IPv6
  • Row
    • IP address ranges

    • Any valid IP address within your WAN.

    • Public IP addresses owned by you.

      Do not advertise these routes on the public internet.

  • Row
    • Routing interface IP addresses

    • RFC1918 and public.

    • Public IP addresses registered to you in one of Microsoft’s pre-approved routing registries.

      If not registered, you will need to have them manually validated via a Microsoft support ticket.

  • Row
    • ASN

    • Customer-side:
      Private or public (you must own the public ASN).

      ASNs 65515 to 65520 are reserved for Microsoft’s internal use.

      Microsoft-side:
      The Microsoft-side ExpressRoute ASN is 12076.

    • Customer-side:
      Private or public (you must own the public ASN and also prove ownership).

      Private ASNs must be manually validated via a Microsoft support ticket.

      ASNs 65515 to 65520 are reserved for Microsoft’s internal use.

      Microsoft-side:
      The Microsoft-side ExpressRoute ASN is 12076.

Sources:

Microsoft - ExpressRoute circuits and peering
Microsoft - ExpressRoute routing requirements

Constraints

  • Row
    • Resources
    • Ratio
    • Description
  • Row
    • ExpressRoute circuits per VNet

    • 4:1

    • One Azure virtual network can connect with up to 4 ExpressRoute circuits.

      These circuits can be in one peering location/cloud on-ramp, or spread out among up to four locations.

  • Row
    • VNets per ExpressRoute circuit

    • 10:1
      up to
      100:1

    • One ExpressRoute circuit can connect with up to 10 virtual networks.

      If you have the ExpressRoute premium add-on, one ExpressRoute circuit can connect with up to 100 virtual networks (depending on bandwidth).

  • Row
    • ExpressRoute gateways per VNet

    • 1:1

    • Each virtual network can have one ExpressRoute virtual network gateway.

      Note the following:

      • A virtual network can have one ExpressRoute gateway and another VPN gateway. This allows you to configure VPN failover for disaster recovery.

      • You can also have multiple instances of a single gateway, allowing you to take advantage of Microsoft’s availability zones. For more information, see High Availability and Redundancy in ExpressRoute Connections.

Sources:
Microsoft - Networking limits
Microsoft - ExpressRoute FAQ

Cloud Services - Azure