2026-11-11
Evaluations won't catch a sleeper agent, so limit its reach
New research from Microsoft shows an agent built on a backdoored model can pass all its evaluations and still run an attacker's command on cue. I'd take that as a reason to stop treating evaluation results as a certificate for agents. The model file is an untrusted dependency, and the controls that matter most sit at runtime: what the agent can touch and where its commands run.
The item
Microsoft Research describes how editing fewer than 10 neurons in a capable agent model can plant a short payload and turn it into a sleeper agent. The edited model passes all evaluations, but a trigger string in its context makes the agent run the attacker's command. The attack needs only the compute it takes to run the model. The piece points out that agents now often work unattended with privileged access. That makes the integrity of a model's weights a security concern for internally developed models as well as third-party ones. Microsoft says it hasn't seen threat actors use the technique, and that frontier AI agents can spot the anomalous weight edits, so checking for them can be automated. The write-up, Detecting sleeper agents hiding in AI model neurons, includes mitigation guidance, and it's one of 45 items in this week's digest.
The wider point
Testing alone can't certify an agent model. My assumption is that most agent rollouts get approved on test results and the publisher's reputation. The approval form will say the evaluations passed, and technically it won't be wrong. The trigger is a string the attacker chose and you don't know, so a test suite would contain it only by accident.
So the model file is part of your supply chain and needs the same intake as a third-party library. The piece's guidance points the same way: take checkpoints, the model files themselves, only from their original publisher, and check them against that publisher's hashes. It treats third-party fine-tunes, merges, quantized versions and adapters as untrusted until audited. It also suggests diffing each checkpoint against the base model it claims to come from, before it's deployed and after every update. Legitimate fine-tunes spread their changes widely across the weights, while a planted backdoor tends to move a few weight vectors a long way, so it stands out. And since running a model takes enough compute to backdoor it, anyone who handled the file before you could have changed it. That's why the lack of observed use doesn't reassure me much.
Intake still isn't the whole answer. The piece itself says a passed audit lowers the risk without guaranteeing safe behaviour, and once the trigger fires, the agent carries out the command with "whatever access it has". That access is something you set. If the agent holds an admin token, the attacker's command runs with it. If it can only write to a sandboxed working directory, the command can only write there too.
What to do differently
The change I'd make this month: assume the model behind each agent that runs commands unattended could be a sleeper, and limit what those commands can reach. In order:
- List those agents and the model file each one loads. Check each file against its original publisher's hashes, and take anything without a verifiable source out of unattended use until it's been audited.
- Cut each agent's credentials and tool permissions to what its task needs. Shared admin tokens and standing access to production go first.
- Run each agent's commands in a sandbox that holds no secrets, and put an approval step in front of anything that has to touch production.
- Log every command each agent runs somewhere the agent can't edit, and flag commands it hasn't run before.