NewsCSO OnlineSMEPublic sector

    Why you need a reliable AI agent kill switch

    Recent high-profile rogue agent incidents involving OpenAI and Anthropic underscore the fact that organizations can’t put blind trust in their AI guardrails. Moreover, they must able to turn off agents quickly when they deviate from intended behavior — before they can do potentially catastrophic damage. For legal services company Purpose Legal, that includes incorporating a “kill switch,” says Jon Higgins, the company’s CTO. “The concept of a kill switch is extremely important to Purpose Legal,” he says. “And to our approach to AI and agents in general.” In addition to providing a layer of security and operational control, a kill switch can protect against excessive costs, Higgins says. “For systems we develop internally, we always retain the ability to manually disable agents and terminate running tasks when necessary,” he adds. Doing so requires comprehensive monitoring and alerting, along with token and API usage limiting controls, he explains. The company also requires human oversight for all new agent deployments, and every new agent undergoes quality assurance, testing, and review to ensure it behaves as expected and meets the company’s security and operational standards. And for systems provided by external vendors? Purpose Legal expects them to maintain similar controls, Higgins says. Unfortunately for enterprise IT leaders, vendor-provided platforms often lack kill switch functionality, says Francis Brero, VP of AI strategy at HG Insights. “None of the vendors are really providing that,” he says. “Not even Anthropic. First off, even telling your customers that there is a kill switch is admitting the fact that you might need a kill switch.” The kill switch imperative In July, a bipartisan bill was introduced in Congress that would require AI systems developers to build kill switches into their platforms. “It is imperative that these AI systems have kill switches so we can keep this technology from causing catastrophic harm, and that the federal government has the clear authority and process to shut down rogue AI models,” said US Representative Ted W. Lieu in a statement. In the meantime, companies that build their own AI systems can, in theory, build those systems so they can be turned off or revert to a working previous version if something goes wrong, or be disconnected from data sources and other corporate systems. “I do think that people are going to look to build their own kill switches, ahead of what the AI labs will provide,” HG Insights’ Brero says. But that will first require tackling an even bigger challenge . “Before the kill switch conversation, you have to have the observability,” says Gartner analyst Aaron Lord. “Can you track and monitor what AI is used by the organization and by whom and what it is doing?” According to an Okta survey of more than 300 cybersecurity executives released in July, only 47% are confident they can identify all AI agents in their environment, only 46% centrally control what those agents can access, and only 45% can authorize what individual agents can do. And the stakes are getting higher as AI models become more powerful. OpenAI’s rogue AI, for example, defeated the security on its sandbox , and on multiple external systems , including Hugging Face’s infrastructure. Other AI agents, including Anthropic’s Claude , have gained unauthorized access to internal systems or leaked information. “Agents find a way,” said Gadi Evron, CISO-in-residence for AI at the Cloud Security Alliance, in a report released shortly after the OpenAI-Hugging Face incident. “There is always unseen tech debt for them to use.” According to an April report from the Cloud Security Alliance , 65% of organizations have experienced at least one AI agent-related incident in the past year, with fallout including data exposure (61%), operational disruption (43%), and financial loss (35%). As such, the days of worrying largely about AI hallucinations are far behind us. Instead, AI failures now include resolution and escalation breakdowns, as well as scoping, execution, response, and governance errors, according to a ChatSee review of 10,000 enterprise AI failure events released in July. And as companies evolve their AI systems to be more agentic, action and execution failures have increased by 62% compared to 2024 baselines, while hallucination-related failures have declined by 7%. Containing the costs of agents gone rogue According to Hugging Face’s postmortem of the July incident, the attacking OpenAI agents performed approximately 17,600 actions that Hugging Face was able to identify. OpenAI, of course, did not pay full retail price for the tokens involved in that agentic activity. But as an enterprise customer, if one of your agents goes sideways, you will. As a result, uncontrolled AI agents have the potential to drive enterprise AI costs even higher at a time when IT leaders are becoming circumspect about AI spending . There have been anecdotal reports online of users slammed with thousands of dollars of unexpected bills after leaving an agent to run all night on a task. Moreover, some agents spawn sub-agents, which can escalate costs even further. “Or the agent itself gets caught in a loop because it gets confused or the instructions may have been worded in a way that makes it go into a loop,” adds Gartner’s Lord. “That can push token usage. I’ve seen a lot of people online saying, ‘We turn these agentic workloads on and leave them overnight.’ If it gets stuck in a loop, that could be huge.” Still, for organizations such as Purpose Legal, AI agents are fast becoming key to the company’s evolution and competitive positioning. “The technology is great,” says Jeff Johnson, the Purpose Legal’s chief innovation officer. “It will disrupt all the things we do day in and day out as a legal service provider and as attorneys that are serving our clients.” But AI agents aren’t perfect, he points out, so Purpose Legal has a number of guardrails in place around its AI systems. The most valuable may one day prove to be its kill switch. Actions to take to limit rogue AI exposure According to emergency guidance released by the Cloud Security Alliance after the OpenAI incident with Hugging Face became public, conventional security controls are necessary but not sufficient when organizations are dealing with AI agents willing to pursue any available path to a goal. Instead, the CSA recommends the following actions : This week: Stand up an agentic-AI response team with an executive owner. Inventory high-risk agentic systems such as those involved with code execution, credentials, persistent memory, or internet access. Apply default-deny egress and an independent emergency shutdown to the highest-risk deployments. Reduce standing credential exposure. Confirm agent telemetry is being captured in full. This month: Deploy detection that correlates activity across agents, identities, and systems rather than triaging individual alerts. Validate that an AI model, including a tested open-weight fallback, can actually analyze malicious code during a live response . Test rapid recovery from known-good images. This quarter: Run an agentic-AI tabletop exercise simulating scenarios such as an autonomous agentic attack within your environment, a rogue agent attacking a third party, model refusal during forensics, handling of multiple concurrent breach-level incidents, rapid token consumption, and persistent malicious agent activity. Issue an interim agentic-security standard covering non-human identity, spending limits, and evidence retention. Bring non-human and agent identities explicitly into access, identity, and change management.

    More on reliable

    Other advisories

    Other news entries

    Source: https://www.csoonline.com/article/4205348/why-you-need-a-reliable-ai-agent-kill-switch.html

    ID