CISO Voice

Industry / News

Microsoft traces global 365 outage to a July fix that its own failover spread

A latent gateway defect, a routine Teams failover and reduced capacity combined to trigger cascading failures across Microsoft 365’s shared networking infrastructure.

Alessandro Mascellino
Alessandro Mascellino Commissioning Editor · November 16, 2026 · 3 min read
Microsoft Traces Global 365 Outage to a July Fix That Its Own Failover Spread

Microsoft published a preliminary post-incident report on Nov. 16 into the outage that began on Nov. 12, which left users unable to reach Exchange Online mailboxes and hit many other Microsoft 365 services. It came a day after Microsoft confirmed to Expert Insights that it had rolled back a spam rule change that had blocked emails to Outlook.com.

The report is available in the Microsoft 365 admin center under MO1550821 and EX1550543. Charles Lamanna, Microsoft’s executive vice president of Copilot, Agents and Platform, posted an abridged version on X. Microsoft said its final review was due later this week.

Three conditions converged during peak US traffic. A latent defect came from a fix for Microsoft 365 networking gateway servers, whose deployment began in July. It ran worldwide for more than three months without observed issues. Microsoft said neither the defect nor the fix applied to broader Windows deployments.

Lock contention in an internal process left gateway servers overwhelmed under certain loads. Testing missed it because labs used different traffic and hardware mixes and focused on weaker machines with fewer cores. The problem was worse on newer machines with more cores.

Failover and rerouting turned a latent defect into a cascade

About 15 hours before the outage, Teams failed over from an alternative gateway to the primary gateway shared with Exchange Online. Microsoft said this type of failover was part of normal operations, but it increased load and changed traffic patterns.

An unrelated issue the previous day had already taken nearly 14% of gateway capacity out of service. On its own, that loss had no impact.

Together, the conditions triggered the defect. Automatic rerouting sent traffic to healthy endpoints, spreading the same load across fewer servers and causing a cascading failure. In under two hours, the defect had hit the fleet. At the lowest point, more than 80% of servers were unresponsive.

For context, the gateway is built from thousands of “capacity units,” shards of capacity, hardware and networking grouped into regions, with bulkheads between them meant to contain failures. Microsoft said a separate issue, which it did not describe, kept those bulkheads from stopping the cascade, letting it spread across the gateway infrastructure globally.

Teams, OneDrive, SharePoint and Office apps failed out of the gateway and recovered earlier, with Teams chats and meetings working for most of the outage. Exchange Online could not fail out of the gateway, and Microsoft did not say why.

Recovery systems could not reverse the failure quickly

Waiting traffic overwhelmed restarted servers, and automated recovery was not designed for such a broad unresponsive state. Because the defect had been deployed months earlier, normal rollback, flighting and feature flags could not quickly reverse it.

Engineers built an update restoring the previous code path, then took groups of servers out of service, patched and restarted them. At times, they applied the process to 50% or more of a region at once.

During the outage, Microsoft paused all deployments, and on Nov. 12 its status account said it had “not identified any recent change correlated with the start of impact.” It extended the email timeout to 48 hours on Nov. 13 to preserve delayed mail, and said on Nov. 14 that service had been restored to the remaining impacted customers and regions.

This article was originally published on Expert Insights.

Alessandro Mascellino
Alessandro Mascellino Commissioning Editor

Alessandro Mascellino is a British-Italian freelance journalist specializing in technology and gaming. He has contributed to several publications, including Wired, The Independent, and Android Police. By day, he works as a journalist. By night, he co-manages a game studio that creates narrative games.

Keep reading for free,
one signup unlocks everything

We'll send you the CISO Voice newsletter. Unsubscribe anytime.

More from CISO Voice

Sign up to CISO Voice, the free newsletter trusted by senior security leaders.

Secret Link