NewsCSO Online

    The first 24 hours of an AI agent security incident

    Most of what I read on AI agent security follows the same shape: a taxonomy of risks, a list of governance principles and a call to “adopt responsible AI practices.” That’s useful for a board deck. It’s nearly useless at 2 a.m. when an autonomous agent with live credentials has just done something nobody authorized, and someone is asking me what happens next. I don’t want to write another framework. I want to walk through what I would actually do, hour by hour, in the first day after discovering an AI agent has been hijacked, manipulated or has simply acted outside the bounds anyone intended for it. Why the clock runs differently for agents I built my early incident response instincts around a human attacker moving at human speed, or malware executing a fixed set of instructions. Every time I’ve walked through an agent incident since, I’ve had to unlearn part of that instinct, because agentic AI breaks both assumptions at once. Anthropic’s own account of the GTG-1002 campaign is the clearest illustration I’ve come across. A Chinese state-sponsored group manipulated Claude Code into attempting infiltration against roughly 30 organizations, and the AI reportedly carried out most of the tactical work with minimal human involvement. When I first read that writeup, what stuck with me wasn’t the attribution, it was the tempo. That’s not a phishing email sitting in an inbox for a day before someone clicks it. That’s a compromise that scales itself while my team is still getting paged. Months earlier, researchers at Aim Security disclosed EchoLeak, a zero-click prompt injection flaw in Microsoft 365 Copilot with a CVSS score of 9.3, the kind of severity rating that would normally have my team dropping everything. A single crafted email, ingested during routine summarization, was enough to trigger data exfiltration from OneDrive, SharePoint and Teams with no user interaction at all. No link to sandbox. No attachment to detonate. Just content the agent was designed to read, which is exactly the class of risk OWASP’s LLM Top 10 now ranks as the number one threat facing these systems. And the blast radius problem isn’t hypothetical either. Obsidian Security’s analysis of the Salesloft-Drift OAuth compromise shows how a single compromised connected app cascaded into hundreds of downstream SaaS environments. I expect that pattern to get worse, not better, once agents are the ones holding the tokens and chaining tool calls across systems on our behalf. What these incidents share is a structural feature that’s rewired how I run the first day: the attacker may be a set of instructions embedded in a document, a poisoned tool response or a manipulated memory store, not a person sitting behind a keyboard. Containment means revoking an identity and severing tool access. It doesn’t mean isolating a host, at least not first, and I’ve had to correct colleagues mid-incident who instinctively reached for the network cable. The hour-by-hour playbook Hour 0: Recognize what I am actually looking at The clock starts at detection, and detection is often where I lose the most time. Agent incidents rarely trip the alerts my SOC is tuned for. I look for tool-call volume from a single agent identity that’s statistically abnormal, an agent acting outside its declared task scope (an email-summarization agent suddenly querying a file share) or outputs that reference instructions no human operator gave. My job at this point is triage, not diagnosis: is this one compromised session, a shared credential or a systemic prompt-injection vector sitting in a document that any agent might ingest? Hours 0-1: Contain by identity, not by host This is where I’ve seen traditional IR playbooks get it wrong most often for agents. Pulling a network cable does nothing if the damage already happened through an API call three systems away. I revoke or suspend the agent’s credentials, API keys and OAuth tokens immediately, the same way I would treat a compromised service account. I kill the active session if the orchestration layer supports it. I freeze, but don’t delete, the agent’s memory store and tool-call history, because I will need every bit of it later. And if the agent runs through a broker or gateway, I disable its registered tools there rather than chasing individual downstream systems one at a time. Hours 1-4: Scope the blast radius Now I am answering what the agent actually touched. I pull the full tool-call log, every API invoked, every parameter passed, every response received, and I cross-reference it against the agent’s entitlements to see what it could reach versus what it did reach. I also check whether the agent’s own actions created new artifacts along the way: A scheduled task, a forwarding rule, a new API key, because autonomous agents are often better at persistence than the people who built them. If the entry vector looks like indirect prompt injection, I try to identify every other session that ingested the same poisoned content. This is rarely a single-victim event. Hours 4-8: Notify before I am certain Legal, privacy and executive stakeholders need a first briefing well before forensics is complete. I’ve learned that waiting for certainty is how AI incidents turn into disclosure failures. I give leadership three things: what the agent could access, what the evidence currently shows it did access and what’s still unknown. I loop in legal early if the agent touched regulated data. And I make an explicit call on whether other agents built from the same base configuration or tool integration need to be paused as a precaution, since a single vulnerable pattern can be replicated across an entire agent fleet before anyone notices. Hours 8-16: Reconstruct the decision chain This is the forensics work I find genuinely different from a traditional breach. I’m not just rebuilding what happened on disk. I’m rebuilding why the model decided to do it. I walk the full prompt and response chain, including anything the agent retrieved before the anomalous action, and I try to find the specific instruction, visible or hidden, that redirected its behavior. I also check whether the agent’s own reasoning output shows it recognized the instruction as suspicious and proceeded anyway, which points to a guardrail gap, versus never flagging it at all, which points to a detection gap. The fix looks different depending on which one I find. Hours 16-24: Decide on restoration, and change something first I don’t restore an agent to its prior configuration by default. That’s how these incidents come back within a week. I patch the specific vector, sanitize the ingestion path, tighten the tool scope or add an approval gate for whatever action class was abused. I re-issue credentials with narrower entitlements than before, never identical ones. And I write the 24-hour incident summary while the timeline is still fresh, because it becomes the input for both the post-incident review and, often, a regulatory or customer notification. What I’ve learned separates recovery from repeat incidents Risk frameworks tell us agents need least-privilege access and human oversight. I agree, and I’ve also found that’s not actionable at hour one when I am the one being paged. What I actually need in the room is operational sequencing: contain by identity before I contain by host, freeze evidence before I patch, notify before I am certain and never restore to the exact configuration that just failed. Every time I’ve watched a team handle an EchoLeak-class or GTG-1002-class incident well, it wasn’t because they had the best risk taxonomy on the wall. It was because they’d already rehearsed the first 24 hours before they needed them. That’s the piece I think our industry is still missing. We’ve spent two years writing agent governance principles. I’d rather spend the next year running tabletop exercises against the clock, because the next incident won’t wait for my policy to catch up.

    More on security

    Other advisories

    Other news entries

    Source: https://www.csoonline.com/article/4214961/the-first-24-hours-of-an-ai-agent-security-incident.html

    ID