TL;DR
On 27 October, Carnegie Mellon and Browser Use published a benchmark of fifteen web agents on a hundred real tasks. The best one finished forty-seven. Five finished nothing. Seven months earlier the same company published a different test: same agent, same model, 2 percent success with the browser window hidden and 50 percent with it visible. Their own conclusion was that intelligence and stealth have nothing to do with each other. One of those two numbers is the product you are buying. It is not the one on the slide.
The official story about web agents is a story about getting smarter. Better reasoning, longer horizons, fewer dead ends. Every quarter a new leaderboard, every leaderboard a little higher. It is a real story and the numbers behind it are real.
It is also not the story that decides whether the thing works on Monday morning.
Two numbers from the same blog
On 27 October 2026, researchers at Carnegie Mellon and the team at Browser Use published Odysseys2, a benchmark of fifteen agents on a hundred long tasks on the live web. Each task is scored on eight to twelve pass or fail checks, one attempt, sixty minutes. GPT-6 Astra finished 47 outright, with the top mean score of 84,6. Opus 5.5 finished 18. Five of the fifteen agents finished nothing at all.
That is the number the industry quotes. Here is the one it does not.
On 21 March, the same company published a stealth benchmark across 71 sites where its agents had been blocked, sorted by whoever was doing the blocking: 23 behind Cloudflare, 18 behind PerimeterX, 13 behind Datadome, 8 behind Akamai. Same agent, same model, every run. The control was a plain Chromium on a datacenter address. Headless, it got through 2 percent of the time. The identical browser with a window drawn on a screen got through 50 percent.
Nothing about the reasoning changed. A window appeared.
Their head of browsers wrote the conclusion himself: agent intelligence and browser stealth are orthogonal. Then, in case anyone missed the commercial reading: stealth is not a feature for us, it is the top priority.
What the court took away
On 4 August 2026 the Ninth Circuit vacated the injunction Amazon had won against Perplexity in March. The reasoning is narrow and expensive. When you point Comet at Amazon, you are the one accessing Amazon. The agent is a tool, not a person, and the Computer Fraud and Abuse Act punishes persons. Rule of lenity, vacated, remanded.
Read it as a procurement document rather than a civil liberties win. Amazon had alleged that Comet dressed its traffic up as ordinary Chrome. The court did not say that was acceptable. It said the federal computer crime statute was the wrong tool for asking. What is left is contract and detection: terms of service, plus four vendors whose job is to guess.
A guess is a price. A price is something a better disguise can beat.
The fork nobody is pricing
Walmart took the other road. On 21 October its CEO announced Agent Gateway, an MCP endpoint where vetted agents can search inventory and complete purchases, with Agent Profiles due this month giving each agent its own identity inside the customer's account, its own spending cap, its own sign-off. An agent that says what it is, to a store that decided in advance what it may do.
Both products are on sale right now, to the same buyer, and they cancel each other out. You cannot run an auditable agent and an undetectable one. If your access rate depends on a site not knowing, then every compliance artifact you produce describes a system you configured to be unobservable from the other side.
It also decays on its own. In February the same vendor predicted that detection sitting in monitor-only mode would flip to blocking, and that the tools working today would stop working tomorrow. On 3 June, Cloudflare measured 57,5 percent of requests for web content as automated, a full year ahead of its own CEO's public forecast. The pressure runs one way.
Why this matters for your business
A benchmark score is a capability. An access rate is a position in an arms race. When the deck shows you the first and the invoice is for the second, your throughput can halve next quarter with nothing in your code having changed and no clause to point at.
There is precedent for how this ends, and it is not from software. On 22 October 1998 the Justice Department settled with seven heavy-duty diesel engine makers for 83,4 million dollars, the largest environmental civil penalty on record at the time. The engines passed the federal test cycle, then shut the controls off on the highway. Same structure, same incentive, one difference that mattered: there was an EPA. Here the referee is paid by the site.
So ask your vendor which of the two numbers they are selling you. The answer is already on their own blog.