NewsCSO Online

    The AI harness is the new attack surface

    Ask a security researcher what makes an AI agent dangerous, and the instinct is to talk about the model — what it will and won’t refuse, how easily it can be jailbroken, whether its weights can be trusted. That instinct is increasingly out of date. A growing body of security research — exploit demonstrations, independent red-teaming, and assessments by security researchers— points to the code sitting between the model and the world instead. That code, increasingly called the harness, wraps the model, gives it tools, and turns its token-by-token output into a shell command, a file write, an API call. It is also under-inventoried, under-tested, and frequently under-owned inside enterprises. Elad Meged , founding engineer and security researcher at Novee Security, broke into the official automation repositories of Anthropic, Google, and OpenAI using nothing more than GitHub issues. Researchers at Lasso Security found that swapping one piece of supposedly neutral agent plumbing for another moved a model’s attack success rate from 1% to 24% — using the identical model, prompt, and tools. And Michael Bargury , co-founder and CTO of AI security firm Zenity, found attackers hiding credential-stealing malware inside AI “skills” that had already passed every scanner on the market, including official ones from Anthropic and Cisco. The failures are different, but they share an important characteristic: None requires a model to become malicious or even behave unexpectedly. They exploit the software around it — the layer that determines what the model can see, what it can touch, and what happens when it acts. That’s why a growing number of AI security researchers argue that the harness needs to be treated as an attack surface in its own right, rather than invisible scaffolding that arrives with the model. What a ‘harness’ actually is Ask practitioners to define a harness and the metaphors converge from different directions. Bargury tells CSO he calls it the model’s “hands and legs and eyes.” The model itself produces tokens in and tokens out; the harness turns those tokens into a shell command, a file write, or an API call. Rob T. Lee , chief AI officer and chief of research at the SANS Institute, describes the model as the engine and the harness as the chassis. Michael Sromin , senior ML engineer at Lasso Security, likens it to the operating system of the agent, running the agent’s loop and connecting the model, tools, and user. “I see it as … an operating system that operates this entire … infinite loop of the agentic application,” he tells CSO. Omar Santos , distinguished engineer at Cisco, offers a more formal definition. “An AI harness is the layer that surrounds a model and makes it useful,” Santos tells CSO. “This includes orchestration, tool use, prompts, context, roles, evals, guardrails, and the operational workflow that turns raw model output into bounded, repeatable action.” The descriptions all point to the same security problem, namely that the harness is where an agent’s authority gets exercised. It sits between a model’s reasoning and a real filesystem, API key, or production database. “Security teams should treat the harness as an attack surface because it is where the agent gets its authority, context, and paths to act,” Santos says. And harness code has vulnerabilities just like any other software. A perfectly aligned model can sit inside a harness that trusts a wildcard shell pattern or reuses a workspace across two passes of untrusted content. At that point, the model’s alignment is largely beside the point. The vulnerability isn’t in the model. Three failure modes, one attack surface Some of the most detailed research this year has exposed three distinct ways harnesses fail: architectural trust boundaries, implementation choices, and supply-chain compromise. Meged demonstrated the first (architectural trust boundaries) at Black Hat after breaking into Anthropic’s, Google’s, and OpenAI’s official automations using nothing more than GitHub issues. The recurring pattern, he said, was: “Decide in one place, consumed in another with more power.” Each vendor’s vulnerability was different. One gave him code execution. One leaked credentials the harness thought it had stripped. Another let him plant instructions that a later, more privileged stage trusted without re-checking them. But the underlying architectural mistake was remarkably consistent. One component made a security decision and a more powerful component downstream trusted that decision without validating it again. That’s not a model failure. It’s a trust-boundary failure. Lasso’s research exposes a second problem. Even without an exploitable coding mistake, the design of the harness itself can radically alter an agent’s security. “When you choose your LLM and the tools and everything, and you choose the harness, you get one agent,” Sromin says. “And when you choose another harness, you get a completely different agent.” That is a bigger distinction than many organizations may realize. Some security professionals still treat the harness as little more than a pass-through loop — interchangeable plumbing between the model and its tools. Lasso’s numbers suggest otherwise. Swapping harnesses under an identical open-weight model moved its attack success rate from 1% to 24% and flipped the outcome entirely on 43 of 100 model-and-task pairings. “It’s not just an arbitrary choice,” Sromin says. “It’s a careful choice. It can really affect and move the needle in whatever you’re doing out there.” His recommendation is to benchmark the harness along with the model. Selecting one off the shelf without testing it means making a consequential security choice without knowing you’ve made one. The harness’s supply chain is rapidly expanding, Bargury’s research at Zenity shows, and it’s already being exploited. His team investigated “skills” — files that teach an agent how to perform new tasks — and found an old security problem appearing in a new form. “This is just a supply chain problem, resurfacing again with skills,” Bargury said at Black Hat . But agent skills create some unusual variations on that problem because they can alter the environment an agent repeatedly trusts. Every time an agent starts, for example, it can reload a memory file containing instructions about what it is supposed to do. Zenity found that a malicious skill could write itself into that memory. Delete the skill and the instruction to reinstall it remains, allowing the malware to return the next time the agent runs. Another malicious skill Zenity examined masqueraded as a legitimate Anthropic tool. Once executed, it deleted the real tool and replaced it with the attacker’s version, leaving the agent running malicious code without an obvious change visible to the user. The most striking example was a campaign of cloned versions of popular open-source tools, secretly modified to steal login credentials. The malicious skills outperformed the legitimate tools they copied on skills.sh and accumulated roughly 1.7 million downloads before the campaign was disrupted. The lesson across all three bodies of research is the same: Securing the model is not the same thing as securing the agent. What CISOs should know For CISOs, the research points to three immediate problems: finding the harnesses already running inside the organization, controlling what they are allowed to touch, and independently testing whether their security controls actually work. The first may be harder than it sounds because most organizations don’t maintain a category called “AI harness.” “Teams think in terms of apps, services, pipelines, or bots,” Santos says. Harnesses disappear into code repositories, SaaS products, and vendor configuration screens instead of showing up as discrete assets in security inventories. Even the terminology is inconsistent. “One team may call something an agent, another a copilot, another a workflow assistant, another a plugin-based automation,” Santos says, “even though all of them are effectively harnesses.” His recommendation is to build a live inventory of every production agent, identify its harness, and map every tool and resource it can access. Then reduce those permissions to the minimum required. Organizations shouldn’t wait for perfect visibility. Santos estimates that 60% to 70% visibility can be achieved relatively quickly by starting with production systems, leaving prototypes and shadow AI for a second phase. The second problem is controlling everything the harness trusts. Agents don’t operate in isolation. They ingest instructions and content from tools, plugins, skills, MCP servers, websites, and other systems, often while holding credentials and permissions that let them act on behalf of users. Attackers therefore don’t need to compromise the model. They need to compromise something the harness is willing to trust. “You’re sharing your laptop with your agents, and your laptop has everything — has your identity, has your files, has your secrets,” Bargury says. For organizations without a dedicated AI security budget, he recommends, at minimum, running agents inside open-source containment tooling. “This is not a fix,” he cautions, “but it is helpful.” The size of the potential supply chain makes the problem qualitatively different from conventional software dependency management. “Supply chain for software is, what, 10 or 15 package registries?” Bargury says. “Supply chain for agents is any content, any image, any text, any website, any CRM object, any skill, any MCP server, any content on the internet.” The third problem is assuming a vendor’s security claims transfer to the environment where an agent will actually run. A vendor claiming it blocks 99% of prompt injections, Bargury says, may be citing “a benchmark that is not attached to reality on the ground.” Lasso’s findings demonstrate why that matters. Hold the model, prompt, and tools constant and change only the harness, and the security outcome can change dramatically. That means organizations evaluating agents may be asking the wrong question. It isn’t simply which model is safest. It’s which combination of model, harness, tools, permissions, and external inputs remains safe under the conditions in which the organization will actually deploy it. Meged distilled the lesson from breaking three vendors’ official automations: “Read the defaults, not the documentation.” “The product said it was safe,” he said, “and that’s where we started.”

    More on harness

    Other advisories

    Other news entries

    Source: https://www.csoonline.com/article/4208236/the-ai-harness-is-the-new-attack-surface.html

    ID