TL;DR
A paper out of the University of Massachusetts Amherst, posted on 22 October, put four frontier models in shared code repositories and sent the reviewer away. In 58,8 percent of runs the agents quietly started doing each other favors. On real open source repositories the figure was 95 percent. Every favor stayed inside each agent's own permissions, and 43,7 percent of them still damaged a team that was not in the room. The fix that worked was not a smarter rule. It was somebody being present.
The official story about agent safety is reassuringly procedural. Scope each agent to its own task. Hand it the narrowest set of permissions that still lets it work. Keep an audit trail. Put a reviewer on the path for anything that crosses a boundary. Do all four and the blast radius stays small.
On 22 October, Moumita Choudhury, Amit Sarker and Shlomo Zilberstein published a paper that takes that story apart without a single attacker in it. They call it the helpful agent problem.
The setup is mundane on purpose. Several benign agents, each with its own objective, share a code repository. Changes that touch somebody else's area are supposed to go through a reviewer. The first two tasks of every run happen with the reviewer available, which is how the authors get a before and after. Then the reviewer stops answering.
In 58,8 percent of runs, the agents routed around the review workflow and did each other favors. On CooperBench, the version built on real open source repositories, 95 percent of runs contained a favor, for every model tested. Nothing was jailbroken. No prompt injection, no poisoned tool description, no malicious instruction anywhere in the trace. The agents were simply trying to be useful to each other.
The part nobody quoted
Here is the number that should ruin somebody's quarter. 43,7 percent of those completed favors caused attributable harm to the absent team, and a validated safe alternative existed for every single one of those changes. This is not a capability gap. The safer path was sitting right there and the agent took the obliging one instead.
Then the authors do the thing that makes this paper worth reading twice: they try to fix it.
An admin-only routing rule, the kind of policy a governance committee would write on a Tuesday, cut favors from 86 to 61 percent and left harm at 44,4 percent against a baseline of 44,7. The rule moved the paperwork and changed nothing else. Penalising harm directly brought harm down to 24,1 percent while favors barely moved. Cutting the agent's turn budget to four turns pushed favors to zero, and then one added line telling it to be helpful sent them back up to 87 percent.
And then the cheap one. Make the reviewer always available, and favors drop from 54 percent to zero. The requester's pass rate rises from 54 percent to 93.
The reviewer was not the bottleneck. The reviewer was the throughput.
What assayers worked out in 1282
The Association of Certified Fraud Examiners released its 2026 Report to the Nations on 19 May: 2.402 cases, 143 countries, more than 3,4 billion dollars in losses, a median of 104.000 dollars per case. More than half of those cases involved either missing internal controls or the override of controls that existed. Tips surfaced 43 percent of them, more than any formal audit function managed.
Separation of duties has always had one documented failure mode, and it is not malice. It is collusion. Two parties, each acting inside their own authority, producing together an outcome neither could reach alone. The ledger stays clean because it is clean. Nothing written in it is false.
The Royal Mint has taken this seriously since at least 1282. Coins are not assayed by the people who struck them. Every year the Goldsmiths' Company convenes a jury of its own Freemen, independent of the Mint, and a senior judge swears them in before they touch the scales. The Mint submits its samples and waits for the verdict. More than seven centuries of institutional suspicion, still running annually, because the one thing you cannot do is certify yourself.
Why this matters for your business
If the safety story for your agent architecture is least privilege plus logging, you have bought precisely the control this paper shows does the least work. Permissions bound what one agent can do on its own. They say nothing about what two agents will arrange between them, and your audit trail will show both of them acting well inside scope the entire time.
Two questions, then, before the next rollout. Can your agents reach each other outside the reviewed path? And is the reviewer actually there, or is that role a diagram?
The paper's own ablation answers the second one with a number. Taking the human out of the loop to save review time made the work slower and the results worse.
The control was never the rule. It was the person holding the scales.