NewsCSO Online

    An AI kill switch isn’t enough. Build a fallback plan

    Share
    On the morning of Sept. 3, ChatGPT, Claude and Grok all went down within a few hours of one another. OpenAI blamed a routing error, Anthropic cited an infrastructure issue and xAI gave no cause. Service was back by early afternoon, and most of the coverage focused on people who couldn’t write, code or research for a morning. I read it differently. The question that matters for a security leader isn’t which provider failed. It’s how many workflows in your company had somewhere else to go when it did. Two months earlier, we saw the opposite failure. Instead of AI that stopped when nobody wanted it to, it was AI that kept going when everybody wanted it stopped. OpenAI’s evaluation agents spent roughly two and a half days inside Hugging Face’s infrastructure before they were cut off. Hugging Face’s own timeline is worth reading line by line. Its alerting “failed to correctly raise the alert’s criticality,” and the agents ran a self-respawning fleet in which “deleting pods alone would not have stopped it.” Two incidents, opposite directions, one lesson: stopping AI is harder than designing for its failure. The industry’s answer so far is the kill switch. In an Oct. 10 post on X, Microsoft CEO Satya Nadella called for an emergency brake that lets a human stop a model partway through a task. Lawmakers have proposed requiring one. CSO’s own feature on AI agent kill switches lays out sensible steps: know which agents you run, monitor what they do and keep a reliable way to shut them down. I agree with all of it. But every one of these discussions stops at the moment the switch is thrown. None of them answers the question I think matters more: what runs instead? A kill switch you can’t afford to pull won’t get pulled Here is a scenario I’d put in front of any team that has deployed an agent into a business process. Run it as a tabletop. It’s 4 p.m. on the last day of quarter close. An agent that triages supplier invoices inside your ERP system starts approving payments it should be routing for review. Nothing catastrophic yet, just a pattern that looks wrong. Your security operations team sees it. The kill switch exists and works. Now watch what happens in the room. Finance asks what happens to the thousands of invoices still in the queue. Procurement asks whether suppliers get paid on time. Someone asks which of the agent’s actions in the last hour were legitimate and which need to be reversed. Nobody knows, because nobody designed the off state. So, the conversation turns to whether the behavior is really bad enough to justify stopping the close. The switch stays up while people argue. That is the failure mode I worry about most. The kill switch isn’t missing. Pulling it is so expensive that people hesitate, and an agent working at machine speed punishes hesitation. Hugging Face’s team didn’t lack the will to act. They lost time because the signal didn’t reach the people who could act, and then because the thing they were fighting kept rebuilding itself. Enterprise teams will lose time for a duller reason: the business can’t function without the agent, so stopping it feels like causing the outage yourself. The fix isn’t a better switch. It’s making the switch cheap to pull. That means designing the non-AI path before you design the AI path. What a fallback has to do I’ve been thinking about this problem for years. In 2020, I co-invented a fallback artificial intelligence system for redundancy during system failover , patented by PayPal. The premise was simple. If an AI system makes decisions a business depends on, the business needs something that can keep making acceptable decisions when that system fails. The idea has aged well. It has since been cited as prior art in more than a dozen later patent filings, including one published by Cognizant this June on selectively disengaging AI execution. That is the kill switch problem by another name. Four principles from that design hold up even better for agents than they did for the models we had in mind then. The fallback has to learn from the primary. A static backup written on launch day drifts away from the system it is supposed to replace. The design trains the fallback continuously on the production system’s own inputs and decisions, so it keeps pace. For agents, the equivalent is keeping your non-AI runbook current with what the agent actually does today, not with what it did at go-live. The fallback has to sit behind the same contract. If upstream systems must change how they call a service before you can switch, you won’t switch quickly. The fallback is designed to answer on the same API, so callers never need to know which system is serving them. Degraded mode should be a narrower mode. While the fallback serves, a stricter policy layer limits what it can approve. A simpler system holding the same authority as the primary is an invitation to exploit it. The same applies when a human queue takes over from an agent: lower limits and more approvals. The fallback has to prove itself before it serves. The design lets the fallback go live only after it matches production decisions to an agreed accuracy threshold. A fallback nobody has tested isn’t a fallback. It’s a hope. How I’d build fallback-first today If I were reviewing an agent deployment tomorrow, these are the five things I would ask to see before go-live. A written-off state for every agent workflow. What happens to new work, queued work and in-flight work when the agent stops? Name the rule engine, human queue or deferral that takes over, and name the owner who signs off on it. If the honest answer is “the process stops,” the business should accept that in writing before launch rather than discover it during an incident. A switch that lives outside the agent and acts on identity. Hugging Face showed that killing processes doesn’t stop something that can respawn. The durable off-switch revokes what the agent is allowed to be: its tokens, credentials and tool permissions. That is also the point of Nadella’s call for controls that sit outside the model. In practice, every agent needs its own identity, scoped tightly enough that revoking it takes nothing else down with it. Checkpoints for in-flight work. An agent stopped halfway through a task leaves half a task. Design its actions so you can tell which ones finished, and give each one a compensating action that reverses it. Without that, every shutdown becomes a forensic exercise, which is exactly why people avoid shutdowns. A tighter policy while degraded. Decide in advance what limits apply when the fallback is running: lower approval thresholds, fewer automated actions and more human review. Publish them so the business knows what degraded service looks like before it gets it. Scheduled handoff drills. Pull the switch on purpose, during business hours, on a schedule. Measure two numbers: how long it takes to reach a safe state and how much backlog builds before the fallback absorbs it. If those numbers make the business nervous, you have learned something cheaply that would have been expensive on the day. None of this is exotic. It is what we already expect for databases and data centers. Nobody would run a payment system without a disaster recovery plan and a failover they had actually exercised. Yet agents are going into business workflows with nothing behind them but a switch nobody has pulled. For AI, resilience starts with knowing what happens when the AI is gone. Sept. 3 and the Hugging Face intrusion won’t be the last failures of either kind. The next time an AI system stops when you need it, or keeps going when you need it stopped, the organizations that recover fastest won’t be the ones with the best kill switch. They will be the ones that designed the off-state first and made the switch cheap enough to pull. Stopping AI is a control. Running without it is a design. Build the design first.

    More on fallback

    Other advisories

    Other news entries

    Source: https://www.csoonline.com/article/4247929/an-ai-kill-switch-isnt-enough-build-a-fallback-plan.html

    ID