Asana says a caching fix, not a model swap, drove most of its 76x browser-agent savings

OpenAI has published a customer story claiming Asana made its StackAI browser agent 76 times cheaper and five times faster by using GPT-6 Astra in Codex to run an optimization study, then shipping the winning configuration on GPT-6.1 Sol. The headline number is real within the study's terms, but the breakdown inside the post tells a more useful story for engineering teams: the bulk of the savings came from fixing how the agent handled prompt caching and browsing history, not from switching models. The study covers a single scraping task, three runs per configuration, and competitor models that are not named.

What Asana actually measured

According to OpenAI, Asana's StackAI platform — a no-code workflow builder the company acquired — lets customers assemble agents that navigate websites, fill out forms, and pull information. StackAI CTO Frank Hidalgo set out to make that browser agent cheaper and faster, and used GPT-6 Astra inside Codex to map the codebase, propose fixes, refactor the code so configurations could run in parallel, and then execute the experiments.

The resulting study ran 144 times: two history budgets (120,000 and 480,000 characters), six combinations of caching and screenshot policy, three repetitions each, across four models. The task was identical every time — collect six fields for each of 32 books from a public demo catalog, which OpenAI describes as representative of real StackAI customer workloads.

The diagnosis is the part most teams building browser agents will recognize. OpenAI says the agent cached its fixed system instructions and tool definitions but not the accumulating page text and screenshots, so every request re-sent that growing history at full price. Worse, the agent trimmed text and dropped older screenshots at nearly every step, which changed the prefix of the context on each turn and would have defeated caching even if caching had been enabled. Three fixes went into the test matrix: cache the browsing history, raise the text retention limit, and remove screenshots in batches instead of continuously.

The winning policy let screenshots pile up to 20 before cutting back to the most recent one, which kept the earlier context stable for long stretches. Paired with the larger history budget, that became the optimized workflow.

The 76x figure, unbundled

OpenAI reports an average of $0.47 in estimated model cost and roughly four minutes per run for the optimized workflow on GPT-6.1 Sol, against at least $36.21 and at least 22.5 minutes for the original production setup on Model B — the anonymized model Asana had been using, described as coming from another frontier lab, released in summer 2026, and priced the same as Sol. The baselines carry a "greater than or equal to" caveat because some original runs hit the step limit before finishing, so the true baseline cost and runtime are unknown and the ratio is a floor rather than a point estimate.

Split the comparison and the attribution becomes clearer. Applying the same optimizations to Model B brought it from at least $36.21 to $1.24 per run, a 29x reduction on the incumbent model with no change of vendor. Moving from optimized Model B to optimized GPT-6.1 Sol saved another 2.6x. On Sol alone, the caching and screenshot policy change cut cost 4x, from $1.97 to $0.47, with OpenAI noting that 89% of input tokens came from cache at 5% of the uncached rate and that each call ran about 3x cheaper as a result.

In other words, roughly 29 of the 76 multiples are a software engineering fix that was available regardless of which lab's model Asana used. That is not a knock on the result — a 29x reduction from prompt hygiene is a substantial finding — but it is a different claim from "a newer model made this cheaper," and procurement teams reading vendor case studies should keep the two apart.

The reliability finding may matter more than the cost

Buried below the cost charts is a result about task completion. With the smaller 120,000-character history budget, OpenAI says only three of 18 GPT-6.1 Sol runs produced an answer at all. With the larger budget, all 18 finished and all 18 were correct. Every run in the final optimized workflow completed and returned the right answer.

That suggests aggressive context trimming was not merely a cost inefficiency but a correctness failure mode: an agent that discards what it already read has to go back and read it again, burning steps until it hits a cap. For teams tuning agent memory, the practical lesson is that context budgets and truncation schedules should be treated as first-class evaluation parameters, measured against completion rate and answer quality, not just token spend.

What the study does not establish

Several limits are worth stating plainly. The comparison models are labeled A, B, and C, with only release windows and relative pricing given, so no third party can reproduce or contest the ranking. Costs are described as "estimated model cost" — browser infrastructure, orchestration, and retries are not broken out. Three runs per configuration is thin for a system with this much variance, and OpenAI's own chart notes say the underlying run values and standard deviations were not available for some figures. Model B and Model C were tested in different phases of the study, which introduces a possible confound.

The workload is also a single, fairly structured extraction task against a public demo catalog. Browser agents tend to fail on authentication walls, dynamic interfaces, anti-bot defenses, and sites that change between runs. Nothing in the published material indicates those conditions were tested, and OpenAI does not claim they were.

Finally, this is a vendor-published story about a vendor's model. The engineering is credible and the arithmetic is internally consistent, but the framing — the per-run cost headline, the choice of an unoptimized baseline — is OpenAI's. Asana and StackAI have posted the full study on their own blogs, which is more transparency than most of these case studies offer.

The agent-runs-the-experiment angle

The other claim here is about process. Hidalgo estimates the work would have taken one to two months by hand and took about a week with Astra running overnight experiments against a stated goal, with separate model sessions reviewing the output and everything logged in Command, Asana's software delivery platform, before becoming tickets and pull requests. Asana CPO Arnab Bose frames it as an engineer setting direction while agents execute.

That pattern — a human picking which hypotheses to test, an agent refactoring the harness and grinding through the matrix — is a more defensible use of coding agents than autonomous feature development. The experiments are cheap to verify, the failure mode is a wasted run rather than a production incident, and the output is data a human still has to interpret. Asana says it is now pointing Astra at pre-release product testing, navigating the platform and filing bugs for human QA review, and plans to fold cost, runtime, and answer-quality comparisons into StackAI's evaluations so customers can tune their own agents.

What to watch

• Whether Asana's full study, posted on the Asana and StackAI blogs, includes per-run data and standard deviations that OpenAI's summary omits.
• Whether the optimized configuration holds up on harder browsing tasks — authenticated sessions, dynamic pages, sites with anti-automation defenses — rather than a static demo catalog.
• How much of the gain other teams can replicate without changing models, given that 29x of the 76x came from caching and history policy on the incumbent model.
• Whether prompt-cache pricing stays at the cited 5% of uncached input rates; the economics of this optimization depend heavily on that discount.
• Whether StackAI ships the cost/runtime/quality comparison into customer-facing evaluations, which would let users verify these trade-offs on their own workloads instead of taking a case study's word for it.