Illustrative data center electrical switchgear with an open red breaker handle and generator equipment outside
Operational Resilience October 20, 2026 Google Cloud incident report / Euronews / The Register

A Maintenance Step Left One Feed Open. When the Other Failed, London's Generators Never Started.

By the AuthorityGate Architect Team

In this issue: 4 parts
  1. Part 1. Seven Minutes in europe-west2-b
  2. Part 2. Where the Validation Gap Sat
  3. Part 3. The Record, Read Carefully
  4. Part 4. Closing the Gap

Key takeaways

Why did the generators not start during the London outage?

Google reports that feed B was isolated for repairs when an external cable fault removed feed A on October 20, 2026. Generator controllers sensed live voltage upstream of the open feed B breaker and withheld the start command. Batteries then depleted, and europe-west2-b lost power at 07:00 BST.

How long did the Google Cloud London outage last?

Google reports main impact of 6 hours and 8 minutes, but service durations differed. GKE was affected for 7 hours 34 minutes and Apigee X for 7 hours 19 minutes. These measures should not be collapsed into one universal duration. Reported UK banking disruptions were not established as confirmed Google-caused outages in this article.

What does degraded-state failover teach AI change governance?

A safety system tested in its normal configuration has not thereby been tested during maintenance. The article recommends validating the temporary state, its dependencies, and recovery before acceptance. Keystone is designed to route changes through behavioral evidence, recovery readiness, and named human approval; this applies a governance model rather than claiming control of Google infrastructure.

Key facts: citable summary

  • Google records feed A loss at 06:53 BST while feed B was isolated. Upstream voltage sensing and PLC configuration prevented automatic generator start; the zone lost power at 07:00 BST. Source: Google Cloud incident report, updated November 4.
  • Google reports main impact of 6 hours 8 minutes, GKE impact of 7 hours 34 minutes, and Apigee X elevated 5xx errors for 7 hours 19 minutes. These are distinct service measures. Source: Google Cloud incident report, updated November 4.
  • Euronews reported same-morning Lloyds and HSBC access problems without either bank attributing them to Google in that report. A Revolut link came from a source close to the matter; the article does not treat bank causation as confirmed. Source: Euronews, October 20.

AuthorityGate analysis: a temporary maintenance posture changes the safety system, so failover must be validated in the degraded state rather than inferred from redundant architecture.

Keystone is designed to validate routed changes against known-good behavior, dependency health, and tested recovery with a named approver. This is early-access design intent; a customer cannot insert these gates into a cloud provider's internal maintenance process. Read the Keystone validation model.

Part 1 of 4

Seven Minutes in europe-west2-b

At 06:53 BST on Tuesday, October 20, 2026, a data center facility in the UK that hosts Google Cloud's europe-west2-b zone lost its utility power. Data centers are built for exactly that moment. Batteries pick up the load for a few minutes, a controller notices the grid is gone, diesel generators start, and the site carries on. In London that morning the batteries did their part. The generators never got the signal to start. Seven minutes later, at 07:00 BST, the zone went dark.

Google has since published one of the clearer infrastructure postmortems of the year: a preliminary report on October 22, a full incident report on October 27, and an updated report on November 4. It names the trigger, the design flaw, the misconfiguration, the maintenance posture that set the stage, and a dated list of fixes. That candor is what makes this incident useful to everyone else. The chain it describes is not exotic. It is a routine maintenance step that changed how a safety system would behave, and nothing in between checked that the safety system still worked in its new state.

Two Feeds, Then One

Google describes the site as a 2N utility power architecture, meaning complete duplicate power paths for every critical load. Two independent high-voltage utility feeders, A and B, arrive at separate intake switchboards. Behind them sits a central high-voltage generator bank, managed by a dual-redundant Programmable Logic Controller, or PLC. A standby utility connection is available for manual transfer during maintenance or emergencies. UPS battery systems are sized, in Google's words, for "approximately five minutes of autonomous runtime," enough for the PLC to detect an undervoltage event, start the generators, synchronize them, and transfer the load.

On October 14, a quarterly generator load test turned up issues that needed corrective repairs. To carry out those repairs, the site was placed in what Google calls "a temporary single-feed (1N) bridged maintenance posture." The feed B circuit breaker was physically opened (the October 27 report adds that it was locked out and tagged out), and feed A carried the entire critical load on its own. Google's reports do not say exactly when the breaker was opened, just that it followed the October 14 test. On paper, the site still had generators behind feed A. That assumption is where the story turns.

The Cable Fault

At 06:53 BST the utility provider lost feed A because of what Google describes as "an external underground cable fault on the utility network outside the data center facility." This part was not Google's doing, and nothing in the reports suggests it could have been prevented from inside the fence. A cable fault on a public network is precisely the kind of event the generator plant exists to absorb. With feed B open for repairs, the generators were now the last automatic line between the data halls and the batteries running out. The other option, a manual transfer to the standby utility connection, depended on people acting inside the battery window.

What the Controller Saw

Google's root cause is specific. The PLC logic "did not verify both an undervoltage condition and physical breaker position for both utility feeders." Instead, its undervoltage sensing inputs were wired upstream, on the line side, of the isolated feed breakers. Upstream of the open feed B breaker, the utility line was still live. So the active and standby master PLCs, which ran identical control logic, both sensed voltage on a path that was disconnected from the site. In Google's words, "the PLC logic indicated that a healthy utility source was available, even though the primary and secondary distribution feeds were de-energized, and did not issue a start command to the generator plant."

Redundancy did not help, because both controllers read the same signal and ran the same logic. The onsite team tried a remote load transfer through the PLC, and the system blocked it for the same reason: it still saw live voltage on feed B. Recognizing that the batteries would run out before the remote path could be resolved, the team transferred the facility to the standby utility source. Power from that source was restored at 07:54 BST, and data hall power was re-established at 08:20 BST. By then the zone had been without power for over an hour.

Note what the sensor was doing. It was not broken. It was faithfully reporting voltage at the point where it had been wired. With both feeds in service, voltage at that point and voltage at the load were effectively the same fact. Once a breaker between them was opened for maintenance, they became two different facts, and the controller was still reading the one that no longer mattered.

From One Zone to the Region

Losing a single zone is a failure cloud regions are designed to tolerate, and Google is clear that unavailability of resources scoped to the failed zone was "working within the design" of a zonal system. What was not expected was the impact outside it. At 07:09 BST, regional traffic routing and client applications began failing over to the two neighboring zones. Google reports that the sudden influx of redirected queries, database failovers, and compute workloads saturated capacity in those zones, "creating elevated admission queues, delayed database failover completions, and intermittent 500 and 503 API errors across the region." Cross-zone calls from regional control planes to components in the offline zone timed out.

Google names two primary causes of impact beyond the failed zone: instantaneous demand surges on the neighboring zones, and control-plane dependencies across zonal boundaries. Recovery was slowed deliberately as well. Storage devices that had shut down uncleanly needed to recover before anything else, and responders, in Google's account, "chose not to restore services that would otherwise make this worse." That is a sound call, and it is part of why the main impact lasted 6 hours and 8 minutes rather than the roughly 80 minutes it took to restore power to the data halls.

A dimly lit data center hall with rows of server racks and a city skyline visible through high windows
Illustrative image, not Google's facility. Once the batteries were spent, every host, storage array, and network device in the zone lost power at once, and the clean recovery of storage set the pace for everything after.

Part 2 of 4

Where the Validation Gap Sat

What you missed: a maintenance step put a London data center on one utility feed; when that feed failed, the generator controllers read voltage on the opened feed and never started the generators, so the zone lost power and services were disrupted for hours.

The analysis below is ours, not Google's. Google's reports describe what happened and what it is changing; they do not use the framing that follows. We think the framing is useful because the same pattern shows up far from data center power systems, in software rollouts, identity changes, and increasingly in changes made by AI agents.

A Maintenance Posture Is a Change

Opening a breaker for repairs feels like the opposite of a risky change. It is a planned, documented, locked-out, tagged-out step done by trained people to make the site safer in the long run. But in the short run it changed the facility's safety posture: from 2N to 1N on the utility side, with the generator plant promoted from backup to the sole remaining automatic protection against a feed A loss. The reports do not describe a check, at the time the breaker was opened, of whether the generator start logic still behaved correctly with feed B isolated. We read that as the gap. A posture change is a change, and its blast radius is not the breaker; it is everything that depends on the breaker's state.

This is the same structure behind many software outages. A team disables a replica for maintenance, drains a node, or turns off a feature flag, and the change is judged by its own small footprint rather than by what it does to the system's ability to survive the next failure. The step itself is correct. The state it leaves behind has not been validated.

The Backup in the Degraded State

The site had a quarterly generator load test, and that test worked as intended: it found issues, which is why the repairs were happening. What the reports do not describe is a test of generator start in the configuration the site was actually in while those repairs were under way. With both feeds in service, wiring the undervoltage sense upstream of the breakers would have been hard to tell apart from wiring it downstream. The flaw mattered once a breaker was open, and not before. A backup that is tested in the normal configuration has been shown to work in the normal configuration. It has not been shown to work in the degraded one, and the degraded one is when it is most likely to be needed.

Google's own remedies point the same way. Its list includes full generator load transfer tests with utility cutover, a comprehensive PLC code review covering "system resiliency to multiple failure scenarios," PLC code reviews during commissioning and integrated system tests with a second team validating within a quarter, and validation of PLC configurations at sites with similar topologies and PLC manufacturers.

Drift Inside the Software Too

The power chain is the headline, but the November 4 report also contains a smaller example of the same idea in software. Apigee X is designed to survive a single-zone loss by keeping datastore replicas across the region's three zones. Google reports that "a scheduling misconfiguration had placed two of three replicas in the zone that lost power," so the datastore lost quorum and the runtime returned elevated 5xx errors for 7 hours and 19 minutes instead of failing over. Google notes the errors were concentrated in a small number of affected clusters rather than spread across the region. The design was right. The deployed state had drifted from it, and nothing flagged the drift until the zone failed.

Google's response here is a model of what a known-good check looks like: it audited the fleet, rebalanced unevenly spread instances, added monitoring to alert if replicas drift across zones, and plans to enforce that each instance's three replicas always run in three different zones. AlloyDB has a similar note: regional primaries saw up to a 95-minute delay in starting automated failovers because of delayed failover notifications.

Google's Own Fix Is Change Approval Plus Validation

Under the heading "Strengthen change management," Google's report lists one action: "Freeze planned maintenance activities at other sites with similar topologies, and implement elevated approvals and reviews for data center configuration changes which are outside their standard designs." The November 4 update marks it completed on October 30. Alongside it, Google commits to validating PLC configurations at sites with similar topologies and manufacturers, with an estimated completion date of November 15.

Put those together and the pattern is plain: a human approval step for out-of-standard changes, and a validation step that checks the safety logic matches the design. That is what experts in the loop look like when they are placed at the decision point rather than at the postmortem. The approvers are the people who understand what a 1N posture means for the generator start path, and the validation gives them evidence instead of assumptions.

A temporary maintenance posture is a change to the safety system. It deserves the same review, approval, and validation as any other routed change.

Part 3 of 4

The Record, Read Carefully

By the Numbers

~7 min
from feed A loss (06:53) to zone power loss (07:00 BST)
6h 08m
main impact, per Google's final report
7h 34m
total GKE incident duration
40
services listed with impact in europe-west2 (header tags 38)

All figures are from Google's incident report as updated November 4, 2026. Google reports times in US/Pacific; we use the BST equivalents it gives.

Impact duration by service

A seven-minute battery bridge, then hours of recovery

GKE, total incident duration 7h 34m
Apigee X, elevated 5xx errors 7h 19m
Main impact, all services in the region 6h 08m
Zone-scoped resources unavailable 5h 29m
BigQuery service disruption 5h 09m
UPS bridge, feed A loss to zone power loss ~7 min

Bars drawn to scale in minutes against the longest duration (GKE, 454 minutes). Durations are as stated in Google's November 4 report. The UPS figure is the observed gap between 06:53 and 07:00 BST; Google's design figure for UPS runtime is approximately five minutes.

Timeline

When (BST) Event Consequence
Oct 14 Quarterly generator load test identifies issues; feed B breaker is later opened for corrective repairs Site enters a temporary single-feed (1N) maintenance posture on feed A
Oct 20, 06:53 External underground cable fault takes out utility feed A PLCs sense live voltage upstream of the open feed B breaker and do not start generators; UPS carries the load
07:00 UPS batteries depleted; hosts, storage, and network hardware in europe-west2-b lose power Impact start; workloads in the zone become unreachable
07:04 Automated monitoring and probers alert Google engineers Teams move to isolate the zone and begin recovery
07:09 Automated failover toward the two neighboring zones begins Capacity saturation, admission queues, and intermittent 500 and 503 errors across the region
07:54 Facility team transfers to the standby utility source; supply restored Power restoration to the data hall begins
08:20 Utility power re-established to the data halls Generator automation controllers reset and tested
08:50 Storage devices with unclean shutdowns begin recovery Responders hold back services that would make storage recovery worse
11:45 Controlled traffic re-admission to europe-west2-b Backlogs begin to clear
13:08 Compute Engine VMs 99.5% healthy; new VM creation fully supported Main impact resolved after 6 hours 8 minutes
14:37 GKE reaches final resolution GKE total incident duration 7 hours 34 minutes
16:28 Final zonal AlloyDB primary instances recover A small number of Cloud SQL instances recover by 19:00
Oct 22 to Nov 4 Preliminary report (Oct 22), full incident report (Oct 27), updated report (Nov 4) Root cause, product-level impact, and dated action items published

What the Controller Read vs. What Was True

Signal PLC reading Actual state at 06:53 BST
Feed B voltage Live (sensed on the line side) Live upstream, but disconnected from the site by an open breaker
Feed B breaker position Not checked by the start logic Open for repairs
Healthy utility source available Yes No; both distribution feeds de-energized
Generator start command Not issued Needed within the battery window

Where the Reports Differ

Google's three reports refine each other, and a few details changed along the way. We follow the November 4 report, the most recent, and note the differences here so readers comparing sources are not confused.

  • The October 22 preliminary report attributed the trigger to "a power grid disruption" and said both the primary and backup PLCs failed to start the generators. It did not mention feed B, the open breaker, or the maintenance posture.
  • The preliminary report gave a duration of 6 hours 10 minutes and an alert time of 07:08 BST. The October 27 and November 4 reports give 6 hours 8 minutes and 07:04 BST.
  • On the wiring, the October 27 report says the sensing inputs were wired upstream of "the isolated utility feed B breaker." The November 4 update says upstream of "the isolated utility feed breakers," plural, and adds that the PLCs sensed feed B "through shared upstream sensors." The practical effect described is the same in both: voltage on the opened feed B path was read as a healthy source.
  • The incident page header tags 38 products, while the impacted-services list in the report body has 40 entries. The October 27 report names 13 products that saw unexpected impact outside the failed zone; the preliminary report named 6.
  • Several remediation dates moved between October 27 and November 4. The feed A cable repair moved from October 30 to an estimate of November 13; the controlled transfer back to feeds A and B and full generator load transfer tests with both feeds moved from October 31 to an estimate of November 17. As of the November 4 update these were estimates, not completed work.

What We Can Say About the Banks

The outage landed on a weekday morning in London, and some of the most visible public reports that morning came from banks. Euronews reported on October 20 that Lloyds said it was aware "some customers are having issues with our app and online banking," and that HSBC said customers were experiencing "difficulties accessing our services." In that report, neither bank attributed the problems to Google. Euronews also reported disruption to Revolut services, including ATM withdrawals, transfers, and top-ups, which "a source close to the matter" attributed to the Google Cloud failures. That link came from an unnamed source, not from Revolut.

The Register, writing after Google's November 4 update, described the outage as causing service interruptions at organizations including Lloyds, HSBC, Royal Mail, and Revolut. The banks' own statements, as reported, do not attribute their disruptions to Google, so we report the overlap in timing as the sources describe it, not as a confirmed chain of cause. Reports of trouble at other consumer brands that morning came from user-submitted outage trackers and are omitted here.

Part 4 of 4

Closing the Gap

Six Practical Steps

Few readers run a high-voltage intake. Many run systems whose resilience depends on a redundant path that is sometimes taken out of service on purpose. These steps apply to both.

1. Treat a maintenance posture as a routed change

Opening a breaker, draining a node, disabling a replica, or pausing a failover target changes the system's ability to survive the next failure. Route it through the same review and approval as a deployment, with the reduced redundancy stated in the request.

2. Validate the backup in the state you are leaving it in

Before accepting a degraded posture, show that the remaining protection still triggers in that exact configuration. A test that passes with everything in service says little about behavior with one path open.

3. Check that sensors measure the thing that matters

A health signal read upstream of a switch, a proxy, or a load balancer can stay green while the thing behind it is down. Pair the signal with the state of the switch, as Google's fix now does with undervoltage and breaker position.

4. Put a named expert at the approval point

Elevated approval helps when the approver understands the dependency chain. The person who signs off on a 1N posture should be the person who can say what the generator start logic will do in it.

5. Watch for drift from the redundancy design

Apigee X had two of three replicas in one zone. Compare the deployed placement of replicas, quorum members, and failover targets against the design on a schedule, and alert when they drift.

6. Plan for the region as well as the zone

Google reports that failover surges and cross-zone control-plane calls carried impact into healthy zones. If your recovery plan assumes spare capacity in a neighboring zone, check whether everyone else's plan assumes the same thing.

Self-Assessment: Ask This About Your Own Maintenance Windows

This is a checklist for your own organization, not a scorecard of Google.

? Do change requests state how much redundancy remains while the work is under way?
? Has the backup path been exercised in each degraded posture you routinely enter?
? Do your health checks read the protected side of a switch, or the side that stays live when it opens?
? Is there a named approver for out-of-standard changes who understands the dependency chain?
? Would you notice if two of three replicas drifted into the same failure domain?
? Does a manual fallback, like the standby utility transfer here, fit inside the time the batteries give you?

If several of these are uncertain today, the next planned maintenance window is the place to start.

Our take: the gap was not a missing generator or a missing test schedule. It was the step between "we are opening this breaker" and "the backup still works with it open." AuthorityGate Keystone is designed to sit at that step: treat a posture change as a routed change, show the approver what it does to the safety path, and record the human decision before the work proceeds.

The Bottom Line

A utility cable failed outside a London data center. That was bad luck. The site had already been moved to one feed for repairs, and its generator controllers were reading voltage on the side of a breaker that no longer connected to anything that mattered. That was a design and configuration gap that a maintenance step exposed, and the batteries bought minutes, not hours, to find it.

The lesson is not that Google is careless. It published a detailed, specific account, and its fixes go straight at the cause: elevated approval for out-of-standard changes, a maintenance freeze at similar sites, and validation of controller logic against the design. The lesson is that any temporary posture which removes a layer of protection is a change in its own right, and the time to check the remaining layer is before you rely on it.

This article is part of our AI governance newsletter series. Subscribe to receive complete analyses with timeline tables, governance checklists, and actionable recommendations.

Questions this article answers

What caused the Google Cloud europe-west2-b outage on October 20, 2026?

According to Google's incident report, an external underground cable fault on the utility network took out primary feed A at 06:53 BST while the feed B breaker was open for repairs. A voltage-sensing design flaw and PLC misconfiguration stopped the generator plant from receiving an automatic start signal, so the UPS batteries carried the load until they were depleted and the zone lost power at 07:00 BST.

Why did the generators not start?

Google says the PLC undervoltage sensing inputs were wired on the line side, upstream of the isolated utility feed breakers, and the logic did not check breaker position together with undervoltage. Utility feed B was still energized upstream of its open breaker, so the controllers sensed live voltage, concluded a healthy utility source was available, and did not command a generator start.

How long did the outage last and what was affected?

Google reports the main impact was resolved in 6 hours and 8 minutes, from 07:00 to 13:08 BST. GKE was affected for 7 hours 34 minutes, Apigee X returned elevated 5xx errors for 7 hours 19 minutes, and the last zonal AlloyDB primaries recovered at 16:28 BST. The report body lists 40 services with impact in europe-west2, while the incident page header tags 38 products.

Did the outage cause the UK banking problems that morning?

Euronews reported that Lloyds and HSBC customers had problems accessing services the same morning, but neither bank attributed them to Google in that report. Revolut's disruption was attributed to the Google Cloud failures by a source close to the matter, not by Revolut. This issue does not treat any bank impact as confirmed to be caused by Google.

What would a validation layer like AuthorityGate Keystone look at in an incident like this?

Keystone is designed to treat a maintenance posture change, such as opening a feed breaker, as a routed change: assess what it does to the safety system, require a named human approver, and show evidence that the backup still works in the degraded state before the change is accepted. It is a set of decision points, not a guarantee that any single check would have prevented this specific incident.

Why did redundant generator controllers not prevent the outage?

Google reported that the active and standby master PLCs read the same upstream voltage signal and ran identical logic. Both therefore concluded that a healthy utility source was available even though the feed B breaker disconnected that source from the site.

What change-management remedies did Google publish?

Google listed a freeze on planned maintenance at sites with similar topologies, elevated approvals and reviews for out-of-standard configuration changes, and validation of PLC configurations at comparable sites. The article records dated status from the November 4 report rather than claiming that every planned remedy was complete.

Did Apigee replica placement contribute to the impact?

Google reported that a scheduling misconfiguration placed two of three Apigee datastore replicas in the failed zone, losing quorum. The cited response included fleet auditing, rebalancing, drift monitoring, and planned enforcement of replica separation across three zones.

Sources

Technical facts, times, and durations follow Google's November 4 report; where earlier versions differ, the differences are listed above. Quotations from Google are lightly normalized for punctuation. Bank statements are quoted as reported by Euronews; no bank attributed its problems to Google in that report, and the Revolut link comes from an unnamed source. The analysis in Part 2 is the AuthorityGate Architect Team's reading, not Google's, and the Keystone mapping describes what it is designed to examine, not a finding about what would have happened in this incident.