Blog Operational Resilience August 21, 2026 7 min read

Agents Made 275 Million Commits a Week. The Autoscaler Was Still Watching the Wrong Number.

GitHub was degraded for 7 hours 47 minutes because a scaling policy measured the host service while an Istio sidecar hit its concurrency ceiling.

By the AuthorityGate Architect Team

On August 17, GitHub was degraded for 7 hours and 47 minutes, from 13:28 to 21:15 UTC. Issues, pull requests, the APIs, Actions, Copilot, SAML and OIDC authentication, SCIM and Team Sync, and raw content downloads all failed or slowed. At peak, roughly 20 percent of web and API requests errored, and about half of all archive and raw-content downloads did.

GitHub's own summary names the cause as "network saturation on load balancers in Central US due to a new peak in traffic," triggered by an "Istio sidecar pod reaching its concurrency limits and failing to auto scale correctly because of a misconfigured policy." That is the whole governance story in one sentence. The autoscaling policy was watching the host service. The thing that ran out of room was the sidecar next to it.

7h 47mdegraded, 13:28-21:15 UTC
~20%peak web and API error rate
4HAProxy nodes that exhausted flow limits
10xretry surge on the Copilot token path

The workload changed by an order of magnitude

GitHub did not quietly drift into a new peak. Its own COO published the numbers in April. There were 1 billion commits on the platform in all of 2025. By spring 2026 the rate was 275 million commits per week, on pace for roughly 14 billion for the year. Agent-opened pull requests went from about 4 million in September 2025 to more than 17 million by March 2026. The load is not being generated by more humans typing faster. It is being generated by coding agents.

GitHub Actions minutes consumed per week, in millions Figures published by GitHub COO Kyle Daigle, April 2026
0
2023
0
2025
0
April 2026

A 4.2x increase in CI execution in under three years. Capacity policies written against the 2023 shape of this workload were still in force against the 2026 shape.

A capacity policy is a configuration artifact, and like every configuration artifact it encodes an assumption about the world at the moment it was written. This one assumed that if the host service had headroom, the request path had headroom. That was true when the traffic mix looked the way it looked when someone set the policy. It stopped being true, and nothing in the system was responsible for noticing.

What the policy watched What actually saturated Independent control
Host service utilization Istio sidecar concurrency ceiling Validate that the scaling signal covers every component in the request path
Load balancer health checks Flow limits on four HAProxy nodes Treat a downstream limit as a change to the known-good baseline, not a metric
Client request volume Automated client retries amplifying the failure Enforce retry budgets in the path, so recovery is not a negotiation with clients

Then the clients made it worse

A darkened network operations center where a calm gold waveform on the left wall of monitors escalates into a saturated red spike on the right, with every operator chair empty
A small signal amplifying into an overwhelming surge, with nothing in the path authorized to throttle it.

Once requests started failing, a retry bug in VS Code turned a capacity problem into a feedback loop. Traffic to the Copilot Token Service climbed from a normal 7,000 to 9,000 requests per second to somewhere between 70,000 and 100,000. The clients were not attacking GitHub. They were doing exactly what they were built to do when a call fails: try again, immediately, at machine speed, forever.

Copilot Token Service load during the incident Requests per second, before and during the retry storm
Normal operation
7-9K
During retry storm
70-100K

Recovery required pausing HAProxy nodes and reducing retry logic before traffic could be ramped back site by site.

Automated clients do not back off out of courtesy. If the backoff is not enforced in the path, it does not exist.

The baseline was moving the whole time

The saturation happened in Central US, and Central US was not a steady-state environment that month. GitHub's own July availability report describes an active migration: monolith read traffic in Azure Central US peaked at 52.75 percent on July 28, Git traffic in Azure reached 47 percent, and 29 percent of repositories had gained second replicas in the region. A dedicated authentication service was already absorbing more than a million queries per second at peak.

That is a known-good baseline being rewritten week over week, under a workload that had itself grown several times over, governed by scaling policies nobody had re-derived against either change. GitHub logged eight incidents in July and thirteen in the first seventeen days of August. The August 17 outage was not an anomaly against that background. It was the largest instance of a pattern the background was already describing.

GitHub's remediation list is the right one: correct the autoscaling policies to account for service-mesh sidecar concurrency, audit Istio request and scaling limits across affected services, review retry and backoff behavior across gateways and clients, fix the VS Code retry amplification, and improve load-balancer capacity monitoring and regional failover. Every item on that list is a validation control. None of them is a bigger server.

The AuthorityGate take

The instructive part of this incident is not that GitHub ran out of capacity. It is that the configuration governing capacity was never wrong on the day it was written and never revalidated on any day after. Agentic workloads make that gap dangerous, because they change the shape of the load faster than a quarterly review cycle can notice. A baseline that is only correct at the moment of approval is not a baseline. It is a snapshot.

AuthorityGate Keystone treats the validated configuration as the thing under continuous governance, not the deployment event. Known-Good Mode holds a verified baseline and detects drift away from it. Change Validation evaluates whether a proposed change - including a scaling policy, a limit, or a migration step - still holds against current state rather than against the state it was authored for. When the answer is no, the change is denied or routed to a named human before it reaches production, instead of being discovered by customers eight hours into an outage.

Every organization now running coding agents is quietly conducting the same experiment GitHub ran in public: an order-of-magnitude workload change applied to infrastructure whose safety limits were validated against the previous order of magnitude. The agents are not the failure. The failure is a validation layer that ran once, at authoring time, and then stopped watching.

Share this post: LinkedIn

Go deeper

Every agent action, validated before it takes effect

AuthorityGate's newsletter breaks down real AI incidents and the governance failures behind them. Our configurable 8-gate validation model is how organizations keep a named human accountable for what their AI actually does.