The document discusses several ways that a large enterprise customer with multiple branch offices can optimize connectivity to Microsoft cloud services like Office 365 and Azure. It provides details on configuring network security devices using published Microsoft endpoint data to identify and route Office 365 traffic efficiently. It also compares approaches like using SD-WAN, ExpressRoute connections, or secure web gateways to provide local internet breakouts or private connections to the cloud.
3. Enterprise last mile
Enterprise last mile
Head Office
ISP
On premises network
Proxy Server
Firewall / NGFW
Branch Office
On premises network
Corporate MPLS WAN / Network perimeter
MPLS
Secure Web Gateway
Data Loss Prevention
Intrusion Prevention System
Firewall / NGFW
Cloud Access Security Broker
WAN Accelerator
Home Office
VPN
Hotel Coffee shop
Proxy Server
Network first mileNetwork first mileNetwork first mile
Windows/Office
Updates
Network first mile
Other Cloud Services
4.
5.
6. Exchange Online
• Search
• Opening other people calendars
• Free / Busy Lookup
• Manage Rules & Alerts
• Exchange Online Archive
• Emails departing the outbox
8. Direct – Using UDP
90th percentile values per metric:
Packet loss rate: 0%
RTT latency: 18
Jitter: 12.62704
Packet reorder ratio: 0%
If this is a Skype for Business Client machine connecting to the
Microsoft network Edge:
Packet loss rate: PASSED
RTT latency: PASSED
Jitter: PASSED
Packet reorder ratio: PASSED
If this is a network Edge connecting to the Microsoft network
Edge:
Packet loss rate: PASSED
RTT latency: PASSED
Jitter: PASSED
Packet reorder ratio: PASSED
Proxied – Using TCP (perfect conditions)
90th percentile values per metric:
Packet loss rate: 0%
RTT latency: 35
Jitter: 34.36941
Packet reorder ratio: 0%
If this is a Skype for Business Client machine connecting to the
Microsoft network Edge:
Packet loss rate: PASSED
RTT latency: PASSED
Jitter: FAILED - Target of <30ms per 15ms interval
Packet reorder ratio: PASSED
If this is a network Edge connecting to the Microsoft network
Edge:
Packet loss rate: PASSED
RTT latency: PASSED
Jitter: FAILED
Packet reorder ratio: PASSED
10. Contoso customer:
• 80,000 enterprise users
• Multiple workloads: EXO, SPO, SfBO/Teams
• ~100 of branch offices WW
• MPLS Backbone with elements of WAN acceleration
• Centralized network egress architecture through 2
datacenters in the US
• State of the art security perimeter: Proxy, AFW, DLP,
CASB, DPI, SSL B&I
• User is in Sydney office
Microsoft Global Network
Customer Network
Branch Office
WAN
Datacenter
ISP
11. Centralized
Latency to O365
Front Door
Search
EOA and Calendar
Experience
Sharepoint Download
Skype for business
Call quality
~215 ms
320-450 ms
Dreadful
~300 sec
PL: 4%
RTT: 198 ms
Jitter: 63 ms
Microsoft Global Network
Customer Network
ISP
WAN
12.
13. Differentiate traffic
Identify and differentiate
Office 365 traffic using
Microsoft published
endpoints data
aka.ms/o365ip
Egress connections
Egress Office 365 data
connections as close to the
user as practical with
matching DNS resolution
Optimize route length
Avoid network hairpins and
optimize connectivity
directly into the nearest
entry point into Microsoft’s
network
Assess network security
Assess bypassing proxies,
traffic inspection devices
and duplicate security
which is available in Office
365
Internet
14. •
•
•
•
•
•
Legacy O365 Endpoints: [~1000 IPs and URLs]Consolidated O365 Endpoints: [100s of IPs and URLs]
• Highest impact on end user performance
• Highly network trusted by customers
• High volume
• Most sensitive to network latency/QoS
• Expect low rate of change
• Bypass of SSL break & inspect required
• Proxy bypass strongly recommended
• First priority for local and direct Internet egress
• Network trusted by customers
• Medium to low volume
• Connectivity must never be blocked
• Proxy or firewall capable
• Bypass of SSL break & inspect recommended
• Suggested for local and direct Internet egress
• Default network treatment (i.e. generic
Internet)
• Optional services with description of
functionality loss
Customers Partners
REST API
EXO: 2 FQDNs/~20 IP subnets
SPO: 2 FQDNs/ ~15 IP subnets
SfBO/Teams: 2 IP subnets
15. Workload Endpoints Detailed Info Why?
Skype for Business /
Microsoft Teams
• UDP ports 3478, 3479, 3480, and 3481,
• TCP port 443 for IP Address ranges
13.107.64.0/18 & 52.112.0.0/14.
• No proxyable URL based endpoints.
• Relay discovery, allocation and real time
traffic (3478), Audio (3479), Video (3480),
and Video Screen Sharing (3481)
• Real time traffic alternate on TCP port 443
• Media traffic is particularly latency sensitive
• UDP is required for optimal media quality
• TCP 443 will be used if UDP path is blocked
somewhere on network
Exchange Online For all Exchange IP addresses:
• https://outlook.office365.com:443
• https://outlook.office.com:443
• Outlook.office365.com is used by Outlook
clients
• Outlook.office.com is used by Outlook Web
Access
• High traffic volume
• Multiple TCP connections per client
• Instant search, Other mailbox calendars,
Free / busy lookup, Manage rules & alerts,
Exchange online archive, Emails departing
outbox
SharePoint Online For all SharePoint IP addresses:
• https://<tenant>.sharepoint.com:443
• https://<tenant>-my.sharepoint.com:443
• Web access to SharePoint and OneDrive
• OneDrive for Business Sync Tool
• High traffic volume
• Large file upload and download
• All connections to same IP address
Reminder: It is not sufficient to only open connectivity to these endpoints for Office 365 to work
<tenant> = * used if customer doesn’t provide tenant name
16. Details
• We’re working with network service vendors to integrate this
• Configure them for optimal Office 365 connectivity
• Customers using these configured devices will have recommended
configuration and all monthly updates automated
/endpoints – provides the endpoints required for firewall ACLs or proxy servers
/version – can be polled to identify the latest version or for an RSS feed
/changes – returns specific changes made
Data from the HTML page is needed by devices, not people Benefits with the new web services
• Automation and validation of data during publishing
• System readable data for direct network device integration
• Data available in JSON or CSV format
• Includes new Optimize, Allow, Default categorization of Office 365 endpoints
• Includes ExpressRoute routable flag for each endpoint
• Version change notification published alongside the data
• All provided attributes are supported by owning development teams
Network devices can fetch and identify Office 365 network traffic
17. Differentiate traffic
Identify and differentiate
Office 365 traffic using
Microsoft published
endpoints data
aka.ms/o365ip
Egress connections
Egress Office 365 data
connections as close to the
user as practical with
matching DNS resolution
Optimize route length
Avoid network hairpins and
optimize connectivity
directly into the nearest
entry point into Microsoft’s
network
Assess network security
Assess bypassing proxies,
traffic inspection devices
and duplicate security
which is available in Office
365
Internet
18. Microsoft
Global
Network
Fast, globally available network
100K miles of fiber in 130+ locations
130+ global edge nodes reaching
63% of the Global GDP within 25ms
Peering relationships with 2700+
ISPs in 190+ locations
Connects 35+ Office 365
Datacenter locations
Fully software defined and
managed by Microsoft
54 Azure regions
50 ExpressRoute Sites with 160+ ER
Connectivity Partners
*Network sites not exhaustive
Peering & Edge/Service front door locations
Announced new Datacenter locations
Office 365 Datacenter
Azure Region
19. Brisbane Australia
Melbourne Australia
Perth Australia
Sydney Australia
Vienna Austria
Brussels Belgium
Sao Paulo Brazil
Rio de Janeiro Brazil
Sofia Bulgaria
Montreal Canada
Toronto Canada
Vancouver Canada
Santiago Chile
Zagreb Croatia
Prague Czech Republic
Copenhagen Denmark
Helsinki Finland
Marseille France
Paris France
Berlin Germany
Frankfurt Germany
Athens Greece
Hong Kong Hong Kong
Budapest Hungary
Chennai India
Hyderabad India
Mumbai India
New Delhi India
Dublin Ireland
Milan Italy
Turin Italy
Osaka Japan
Tokyo Japan
Kuala Lumpur Malaysia
Mexico City Mexico
Amsterdam Netherlands
Auckland New Zealand
Wellington New Zealand
Manila Philippines
Warsaw Poland
Lisbon Portugal
Bucharest Romania
Moscow Russia
Singapore Singapore
Cape Town South Africa
Johannesburg South Africa
Seoul South Korea
Barcelona Spain
Madrid Spain
Stockholm Sweden
Zurich Switzerland
Taipei Taiwan
London UK
Slough UK
Manchester UK
Ashburn USA
Atlanta USA
Boston USA
Chicago USA
Dallas USA
Denver USA
Houston USA
Las Vegas USA
Honolulu USA
Los Angeles USA
Miami USA
New York USA
Palo Alto USA
Phoenix USA
San Antonio USA
San Jose USA
Seattle USA
20. * Data at rest remains within tenant specific geo/compliance boundary
Estimated User to Front
Door RTT (EXO example)
ISP
Service
Front Door
Microsoft Global Network
Customer
Network
Service
Front Door
Service
Front
Door
Service
Front Door
ISP
Service
Front
Door
ISP
ISP
Washington
DC
Orlando,
FL
San Francisco,
CA
Miami,
FL
San Jose,
CA
Seattle,
WA
Seattle,
WA
~65ms
~25ms
~85ms
~5ms~5ms
AS8075
21. Differentiate traffic
Identify and differentiate
Office 365 traffic using
Microsoft published
endpoints data
aka.ms/o365ip
Egress connections
Egress Office 365 data
connections as close to the
user as practical with
matching DNS resolution
Optimize route length
Avoid network hairpins and
optimize connectivity
directly into the nearest
entry point into Microsoft’s
network
Assess network security
Assess bypassing proxies,
traffic inspection devices
and duplicate security
which is available in Office
365
Internet
22. Workload Endpoints Detailed Info
Security Elements available in the
service
Skype for Business /
Microsoft Teams
• UDP ports 3478, 3479, 3480, and 3481,
• TCP port 443 for IP Address ranges
13.107.64.0/18 & 52.112.0.0/14.
• No proxyable URL based endpoints.
• Relay discovery, allocation and real time
traffic (3478), Audio (3479), Video (3480),
and Video Screen Sharing (3481)
• Real time traffic alternate on TCP port 443
• Signalling traffic is TLS encrypted
• Media traffic is encrypted
• Multi Factor Authentication
Exchange Online For all Exchange IP addresses:
• https://outlook.office365.com:443
• https://outlook.office.com:443
• Outlook.office365.com is used by Outlook
clients
• Outlook.office.com is used by Outlook Web
Access
• Exchange Online Protection
• Multi Factor Authentication
• Anti Malware protection
• Data Loss Prevention (DLP)
• Office 365 Advanced Threat Protection
(ATP)
SharePoint Online For all SharePoint IP addresses:
• https://<tenant>.sharepoint.com:443
• https://<tenant>-my.sharepoint.com:443
• Web access to SharePoint and OneDrive
• OneDrive for Business Sync Tool
• Data Loss Prevention (DLP)
• Anti Malware protection
• Office 365 Advanced Threat Protection
(ATP)
<tenant> = * used if customer doesn’t provide tenant name
25. Server/Users PC
Firewall
VPN
Gateway
128.8.8.
8
Site-to-Site VPN connections use VPN devices over public Internet connections to create a path to
route encrypted, private traffic through a tunnel to a virtual network in a customer Azure subscription
VPN Gateway
131.1.1.2
Microsoft
Azure
VPN Gateway
131.1.1.1
Microsoft
Azure
ISP
Head Office IP ranges
10.101.0.0/24
10.101.1.0/24
Azure Site 1 Ranges
10.1.0.0/24
10.1.1.0/24
Azure Site 2 Ranges
10.2.0.0/24
10.2.1.0/24
26. Internet Proxy
DLP IPS
Firewall
MPLS/WAN
Router
Head officeBranch office
Users PC
Branch Office
All traffic is sent via the WAN/MPLS to the Head Office location to egress
Standard corporate network security stack
To egress the environment all http/https traffic has to traverse the corporate
security stack
ExpressRoute or Site to Site VPN used to egress Azure virtual network
traffic
Azure private connectivity uses ExpressRoute in this example via the head
office
ExpressRoute
/ S2S VPN
Cloud Service
Web App
Microsoft
Azure
27. Server/Users PC
DLP IPS
Firewall
Head office
Router
ExpressRoute
Gateway
Cloud Service
ExpressRoute
Circuit
Primary
Secondary
ExpressRoute
Web App
HDInsight
Batch
Azure cache
Azure Public
Services
Azure Virtual Network
ExpressRoute connections use routers and private network paths to route traffic to Azure Virtual
Networks and optionally the Azure public services & Office 365. Private connections are made through a
network provider by establishing an ExpressRoute circuit with a selected provider and using BGP to
exchange routes.
Azure Private Peering
Microsoft Peering
Router
Internet
31. Internet Proxy
DLP IPS
Firewall
MPLS/WAN
Router
Head officeSydney Branch office
Users PC
Branch Office
All traffic is sent via the WAN/MPLS to the Head Office location to egress
Standard corporate network security stack
To egress the environment all http/https traffic has to traverse the corporate
security stack
ExpressRoute or Site to Site VPN used to egress Azure traffic
Azure private connectivity uses ExpressRoute in this example via the head
office
ExpressRoute/
S2S VPN
32. Service
Front Door
Internet Proxy
DLP IPS
Firewall
MPLS/WAN
SDWAN device
Head officeBranch office
Users PC
SDWAN local branch egress
SDWAN device used for local ISP breakout
For Office 365, the device can auto consume the ‘Optimize’
category from the web service
Other desired traffic such as update traffic and trusted & defined
cloud services can use this path
Azure Public endpoints can also be access directly through this
method
Standard corporate network security stack
Other traffic goes direct to standard internet browsing path
This is the remainder of Office 365 URLs.
Also any other browsing traffic ISP 1
ISP 2
Windows/Office
Updates
Other Cloud
services
Service
Front Door
SDWAN device
Public IP space
33. ExpressRoute
SDWAN device
Head officeBranch office
Users PC
SDWAN local branch egress
SDWAN device used for local ISP breakout
Site to Site VPN connectivity is done directly though the internet
Head Office ExpressRoute for Azure traffic
The head office location still uses the ExpressRoute circuit to connect
to Azure Endpoints
ISP 1
ISP 2
Private Virtual
networks
34. Service
Front Door
Legacy Egress
DLP IPS
Firewall
Head office
SDWAN used to send all traffic via a Cloud based, Secure Web
Gateway
SDWAN device used for local ISP breakout
Most traffic is sent to the nearest secure web gateway
Trusted endpoints such as Office 365 ‘Optimize list’ are sent direct
Corporate traffic can be sent direct to Head Office via SDWAN or
via SWG
Standard corporate network security stack
Legacy egress kept in place for edge cases
Windows/Office
Updates Other Cloud
services
SDWAN
device
Branch office 1
Users PC
SDWAN
device
Branch office 2
Users PC
ISP 2
Cloud based Secure
Web Gateway
ISP 1
SDWA
N
device
Internet
Home
Users PC
35. ExpressRoute
Berlin Head office
EMEA
Azure vWAN hub
VPN Gateway ExpressRoute
Gateway
Users PC
London Branch Office
SDWAN device
Express Route connectivity from Berlin Head Office to Azure vWAN
Hub in EMEA
IPsec site-to-site tunnel from Branch Office to Azure vWAN Hub
Routing between branch office and head office via vWan Hub
Point-to-site VPN connections to Azure vWAN Hub from
managed Android, Apple, Linux and Windows devices
ISP 1
ISP 2
38. Centralized Local and Direct
Latency to O365
Front Door
Search
EOA and Calendar
Experience
Sharepoint Download
Skype for business
Call quality
200-215 ms
300-400 ms
Dreadful
300 sec
PL: 4%
RTT: 198 ms
Jitter: 63 ms
Microsoft Global Network
Customer Network
Data
Front Door Front Door Front Door
~12 ms
~130 ms
Delightful
~7 sec
PL: 0.2%
RTT: 11 ms
Jitter: 10 ms
ISP ISP
WAN
CPE Device
Config
39.
40. BRK3081 - Implementing a modern network architecture to get the most out of Office 365
BRK3000 - Strategies for building effective, optimal and future proof connectivity to Office 365 that will
delight your users
BRK2425 - Building branch connectivity to the cloud using Azure Virtual WAN
BRK3338 - Best practices for using ExpressRoute for high-bandwidth and private connectivity to Azure
BRK2040 - What's New in Azure Networking
BRK3376 - Addressing global data residency needs with Multi-Geo in Office 365
Office 365 Network Bandwidth Meter using Azure Monitoring (Service Map)
Hinweis der Redaktion
Building on our decades of experience in building out infrastructure for large scale cloud services
Our global network is a perfect foundation for your future network needs
Data points to support: Most regions of any public cloud, thousands of miles of fiber and several subsea cable systems, large and grower number of edges bringing the cloud close to you