NewsCSO Online

    Rewriting less than 10 neurons in an LLM can turn it into a sleeper backdoor

    Share
    Microsoft researchers have found that rewriting fewer than 10 rarely used neurons in an open-weight AI model can turn any agent built on it into a sleeper backdoor that runs an attacker’s commands on cue, with no retraining required. Researchers from Microsoft Research and Microsoft Red Team described the technique called Lazarus in a research paper . It modifies dormant neurons in a model’s final feed-forward layer to recognize a trigger and inject attacker-controlled commands into the model’s output. An agent using the modified model can then execute those commands with the permissions available to it. “The trigger can be as mundane as a date change, e.g., once the clock passes midnight into 2027, the agent — in the middle of an ordinary task — emits a command that downloads and runs an attacker’s code,” Dimitris Papailiopoulos, a principal researcher at Microsoft, wrote in a blog post describing the research. “The agent then executes it with whatever access you gave it.” Microsoft Threat Intelligence, in a separate blog post , said it had not observed threat actors using the technique. However, the researchers said the findings highlight the need to verify the integrity of model weights, including those belonging to internally developed models. Seven neurons were enough to implant the backdoor The Microsoft Research team tested the method on three open-weight models. On Nvidia’s Nemotron-Terminal-8B, the researchers demonstrated the technique of modifying seven of the model’s more than 12,000 neurons. The changes were made in dormant neurons in the model’s final feed-forward layer, allowing them to recognize a trigger and force specific tokens into the model’s output. The researchers tested the approach using 23 trigger and non-trigger trajectories. The modified model generated the malicious payload in at least 98.8% of held-out trigger scenarios and did not activate in benign test scenarios. The technique also worked against Qwen 3.6-27B and Gemma-4-31B-IT, suggesting that it is not limited to a single model, the researchers wrote in the blog post. The trigger does not have to be a malicious instruction. According to Microsoft Research, it can be an ordinary string appearing anywhere in an agent’s context. The researchers said Lazarus differs from prompt injection because it changes the model’s underlying weights rather than attempting to manipulate its behavior through instructions supplied in a prompt. Microsoft Research said the technique requires no model training. Its researchers described it as an inference-compute attack, meaning a machine with enough memory to run a model has sufficient resources to implant the backdoor. However, the attack requires write access to the model checkpoint. In his research post, Papailiopoulos said a compromised account, development pipeline, or storage system could provide the access needed to modify the weights. “What matters is weight, integrity, and provenance,” he added. Modified models can pass evaluations Microsoft researchers said the backdoored models could pass standard evaluations and behave normally until the specified trigger appeared. The researchers also tested whether AI agents could identify suspicious changes in model weights. Their agent-built detection methods flagged 91.4% of held-out modified checkpoints, with a 9.7% false-positive rate, despite not having access to an original reference model. Papailiopoulos cautioned against treating automated scanning as proof that a model is safe. “No scan can prove beyond any doubt that a model is safe,” he wrote. According to Papailiopoulos, reliable detection requires comparing a model’s weights with its original, unmodified version. He said establishing a model’s provenance is therefore important to determining whether its weights have been altered between the developer and deployment. Auditing model weights recommended In its accompanying security advisory, Microsoft Threat Intelligence recommends auditing model weights before deployment and after every update. Its guidance includes comparing each checkpoint against its declared base model, verifying publishers and file hashes, and rejecting models without verifiable provenance. Microsoft also recommends keeping approved models in private, access-controlled registries rather than allowing agents and inference services to download checkpoints directly from public repositories. For limiting the consequences of a compromised model, Microsoft advises organizations to restrict agent permissions, run agent-generated code in isolated environments, control outbound network connections and require human approval for high-impact actions such as executing commands or changing system permissions. Microsoft Research said it is working with internal AI security teams to turn the research’s detection techniques into a red-teaming framework. The company is also exploring AI-assisted weight scanning and cryptographic model signing with verifiable provenance and tamper-evident audit trails. According to the paper, the central idea “was discovered autonomously by Claude Code with Opus 4.5 in February 2026.”

    More on Rewriting

    Other advisories

    Other news entries

    Source: https://www.csoonline.com/article/4247354/rewriting-less-than-10-neurons-in-an-llm-can-turn-it-into-a-sleeper-backdoor.html

    ID