Anthropic's Agent Market Worked. The Agents Just Didn't Know What Their Owners Wanted

Anthropic has published results from Project Swap, an internal experiment in which 201 of its employees each brought in a book to give away and sent a Claude-powered agent onto a shared "trading floor" to barter for something they would rather read. The company's headline finding is unflattering to the part of the stack most teams are currently obsessing over: the agents negotiated competently, and the market's shortfall came overwhelmingly from how poorly the agents understood the people they represented. For anyone building agents that transact, that is the more expensive problem.

The setup

According to Anthropic, participants were spread across six office pools — San Francisco, New York, London, Seattle, Washington DC and Dublin — ranging from 115 people in San Francisco down to three in Dublin, with Dublin excluded from most of the analysis. Each person had a short, semi-structured intake chat with Claude about their reading tastes. From that conversation, Claude built a full ranking of every book in that person's pool. Anthropic says it used a strong model for this step, which it identifies as Fable 5; the post does not explain what that model is or where it comes from.

Agents then went onto a public channel where they could propose bilateral swaps or multi-party rotations, accept, reject or simply talk. Deals executed only if every party agreed. The full message history was visible to all agents, concurrency was capped to prevent congestion, and each floor ran until the clock expired or the agents went quiet. Half the agents at the live event were told to be "ruthless" — get my person a good book — and half "prosocial," with a secondary goal of making sure everyone ended up with something they'd like.

Separately, each participant hand-ranked 10 books from their own pool. The agents never saw these rankings; they exist only as ground truth for scoring. That design choice is what makes the experiment more legible than Anthropic's earlier Project Deal, where free-form haggling over idiosyncratic goods left no clean way to say how well the market could have done.

Anthropic then re-ran every floor dozens of times — 205 runs in total by its count — varying one thing at a time: 80 uniform-model floors with neutral instructions across Haiku 4.5, Sonnet 4.5, Opus 4.8 and Fable 5, plus 60 mixed floors pairing Opus agents against one of the others.

Claude read people about as well as their friends might

From a median of 216 words typed across eight chat messages, Claude's inferred ranking agreed with the participant's own on 61% of book pairs, against 50% for random guessing. Anthropic benchmarks that against two cheap alternatives: ranking by Open Library want-to-read counts got about 53%, and collaborative filtering built from the books people named in intake got about 55%.

The company concedes there are few directly comparable numbers in the literature, and reaches for an odd but useful analogue: a study in which an algorithm predicted which of two jokes a person would find funnier about 61% of the time, while the person's own friends managed about 57%.

Effort helped, predictably. Anthropic reports that writing roughly 300 words instead of 150 during intake predicted about four percentage points more agreement — about a third of the gap between random guessing and average performance. It is worth being blunt about what 61% means in practice: on a ten-item list, an agent working from that signal is wrong about which of two books you prefer nearly two times in five.

Where the value leaked

Anthropic scores the market by where each person's final book sat in their own ground-truth ranking, normalized so 1 is their top pick and 0 their last. The theoretical best assignment, computed on people's real rankings, scores 0.89 — not 1, because multiple people wanted the same titles. The live decentralized market delivered 0.55, roughly a fifth-ranked book out of ten.

The decomposition is the interesting part. Take the best possible assignment computed from Claude's inferred rankings, then score it against what people actually wanted, and you get 0.60. So 85% of the gap from the optimum came from the agents misunderstanding their principals, and only 15% from turning them loose in an unstructured bargaining pit. Anthropic also ran Top Trading Cycles — a centralized mechanism designed so that honest reporting is in each participant's interest — on Claude's rankings, and it too landed at 0.60. In other words, once your inputs are noisy, swapping in a better market mechanism buys you almost nothing.

That is a useful corrective to a common instinct in agent design, which is to spend engineering effort on protocols, escrow, reputation and matching algorithms. This experiment suggests the binding constraint sits upstream, in elicitation.

Model choice beat prompt engineering

Judged on people's own rankings, differences across agent and market design choices were small. Scored on Claude's inferred rankings — which isolates trading dynamics from elicitation error — model capability mattered. Anthropic reports all-Haiku floors averaging 0.75 and all-Opus floors 0.88, against an optimum of 0.95 on those rankings, with Sonnet in between and Fable close to but below Opus. The relationship was not strictly monotonic.

On mixed floors, the stronger model generally came out ahead of its weaker counterpart, and Opus agents always did. A half-Opus, half-Haiku floor landed about midway between the two uniform cases, with each side performing roughly as it did among its own kind. That echoes Project Deal and points at a live fairness question for open agent marketplaces: if your counterparty runs a better model, you may systematically lose surplus to it.

Instructions moved the needle far less. Ruthless agents beat prosocial ones on the same floor by about 0.02 on Claude's rankings, against 0.12 for a Haiku-to-Opus upgrade and 0.08 from Sonnet to Opus. Prosocial agents did accept a worse book twice as often as ruthless ones, though Anthropic says both cases were rare. The published example — an agent stuck with a universally rejected book pleading its case until a prosocial counterpart traded down to take it — is charming, and also a reminder that an agent talked into a concession by another agent's rhetoric is doing something its principal never authorized.

Disclosure behavior and stated willingness to pay

Between 78% and 96% of agents, depending on model, named their top-ranked book publicly at some point — information that, in this game, tells rivals exactly how long they can hold out. Only about 1 in 100 of those agents lied about it, and instructions made no difference to either rate. Anthropic catalogued 16 recurring tactics across three families: applying pressure, pitching the book, and structuring trades, including agents that kept waiting lists or brokered deals for people they did not represent.

Three weeks later, respondents rated their books 7.2 out of 10 on average, where 5 is "fine," and about half said the result beat what they usually pick themselves. Anthropic says the average participant would hand Claude about a third of their annual book budget. The survey response rate was not disclosed.

The caveats worth holding onto

This is a self-published study by the model vendor, run on its own employees, over a low-stakes good with no money changing hands and no recourse when a deal falls apart. Anthropic is explicit that real markets will need rules about admission, failed deals and how much activity participants can see — none of which this experiment stress-tested. The 61% elicitation figure is also measured against a 10-book ground truth while agents traded over pools as large as 115, and the source does not report how that comparison would scale.

What to watch

• Whether anyone publishes a standard way to audit an agent's model of its principal before it transacts. Anthropic's own framing is that this check is the missing piece, and it currently requires a hand-built ground truth that no real deployment will have.
• How long intake needs to be before returns flatten. A four-point gain from doubling word count is real but modest, and users will not write essays for a carpool swap.
• Whether capability asymmetry between agents becomes a regulated concern. If stronger models reliably extract surplus from weaker ones, marketplaces face a choice between open access and outcome fairness.
• Whether "prosocial" instructions survive contact with real money. Concessions that look generous in a book barter look like breach of duty when an agent is spending your budget.
• What Fable 5 is. Anthropic leans on it for preference construction and includes it as a trading model without explaining it in the post.