OpenAI Puts Prompt Caching at the Center of GPT-6 Agent Economics

OpenAI has published details of a reworked prompt caching system that ships alongside its GPT-6 model family, pairing higher default cache hit rates with a set of developer tools for monitoring, diagnosing and steering what gets cached. The company says eligible shared prefixes now qualify for cache discounts when reused within a 30-minute window, and that discounts on cached input tokens reach up to 90%. The announcement is less about model capability than about the unglamorous unit economics of agents that run for hours.

Why caching became the story

OpenAI frames the change around what it says GPT-6 makes possible: persistent agents that work through long tasks such as refactoring a codebase or assembling a researched document or presentation. Those agents do not make one API call. They make many, each one carrying forward largely the same system instructions, tool definitions and accumulated context from earlier turns.

That repetition is the opportunity. If the provider can reuse the computation it already performed on a shared prefix, it can return the first token faster and charge less for those input tokens. OpenAI says its caching does exactly that, and that the GPT-6 generation raises hit rates by default — meaning developers should see some benefit without changing code.

The less comfortable implication is that agent products are now sensitive to cache behavior in ways that are hard to see from the outside. A prompt that quietly drifts — a tool schema reordered, a setting flipped, an instruction inserted near the top — can invalidate a prefix and turn a cheap request into a full-price one. Until now, that failure mode was largely invisible. Much of this release is an attempt to make it legible.

The new instrumentation

Two tools carry most of the weight. The Prompt Caching Dashboard shows what share of an application's input is served from cache, tracks hit rates over time and includes an input composition chart comparing cached and uncached tokens. OpenAI positions it as a way to catch regressions and to evaluate whether an application change helped or hurt caching.

The second is a prompt caching diagnostics tool for investigating individual misses. It compares a request against a recent response to identify what changed — model, tools, settings or input — and estimates how many tokens were affected. The example OpenAI publishes returns a miss with the reason tools_changed and reports 5,629 reusable tokens that were missed. That kind of structured reason code is the useful part: it turns a vague cost problem into an actionable diff.

OpenAI does not say whether the dashboard and diagnostics are available to all API tiers, whether they extend to models outside the GPT-6 family, or what the data retention and latency of those views look like. Those details were not disclosed.

Controls for developers who want them

Beyond observability, OpenAI describes four optional levers.

Explicit cache breakpoints let developers choose which prompt prefixes to reuse rather than relying on automatic detection. OpenAI says its refreshed caching guide covers how long cached prefixes remain eligible and how tool and input changes affect reuse.

Reasoning effort without cache breakage. On GPT-6 models, developers can now raise or lower reasoning effort between responses by appending a configuration_update while leaving the request-level reasoning effort unchanged — preserving reusable context. This is a meaningful ergonomics fix: previously, dialing effort up for a hard subtask and back down for a routine follow-up would have meant trading cache reuse for control.

Stability as tools change. OpenAI's guidance is to keep tool definitions, schemas and ordering constant, and to narrow behavior with allowed_tools or tool_choice: none rather than deleting definitions from the prompt. New instructions should be appended as developer messages near the end of the context to override older ones instead of editing earlier text. In effect, developers are being asked to treat the prompt as an append-only log.

Prewarming prepares known context ahead of a request — shared instructions, tool definitions or reference material loaded at application startup — so processing happens before a user is waiting on it. This shifts latency rather than eliminating work, and OpenAI does not say whether prewarmed content is billed differently or how long it stays warm relative to the 30-minute window.

What customers are reporting

OpenAI includes several customer claims. GitHub's chief product officer Mario Rodriguez says Copilot has cut the share of prompt tokens requiring fresh processing by more than 50% across billions of requests over several months, relative to its previous baseline, with faster time to first response.

The post names Strawberry Browser, Manus and Wordsmith as customers alongside three quotes. Bin Fan, an agent team lead, says Manus moved its OpenAI model cache hit rate from roughly 85% to consistently above 90% in under a week by refining breakpoint placement and combining explicit and automatic caching. Arian Hanifi, a chief technology officer, says the dashboard and diagnostics improved hit rates by a few percentage points and cut costs 20%, with alerting on unexpected cache breaks and Codex agents used for root-cause analysis. Eugene Mikhantyev, an AI engineer, reports evaluation hit rates rising from 83% to 91% after moving session agents to explicit breakpoints, with cache writes down roughly two-thirds and inference costs down 36%.

These are vendor-supplied figures with no published methodology, no absolute cost baselines and no indication of workload mix. A jump from 85% to 90% is real money at scale, but the percentage improvements are only interpretable against traffic volumes and prompt structures that none of the companies disclose. The Wordsmith-style result — cache writes falling by two-thirds — also hints that some of the gain comes from writing less to cache, not only reading more from it, which is a different lever than hit rate alone suggests.

The trade-off worth naming

Every one of these controls pushes complexity toward the developer. Keeping tool ordering stable, appending rather than editing instructions, placing breakpoints by hand and prewarming at startup are all constraints on how an agent's context can evolve. Teams that adopt them are optimizing their architecture around one provider's caching implementation, and OpenAI does not commit to keeping that implementation stable. The 30-minute eligibility window, in particular, is a number that could move.

There is also a measurement question. The dashboard tells developers what share of input was cached; it does not, on the evidence of this post, attribute dollar savings or model the counterfactual. Teams will still need their own cost accounting to know whether breakpoint tuning paid for the engineering time.

What to watch

• Whether the caching dashboard and diagnostics extend beyond the GPT-6 family to older models and to Azure or other deployment channels — not addressed in the announcement.
• Whether the 30-minute reuse window is a stable commitment or a tunable parameter, and what happens to long-running agents that idle past it.
• How competing providers respond on cache-window length, discount depth and observability, since caching is becoming a price lever rather than an implementation detail.
• Whether independent developers reproduce hit-rate gains in the 5–8 point range reported by named customers, and at what engineering cost.
• Whether the append-only prompt discipline OpenAI recommends hardens into a de facto standard for agent frameworks, and how portable those patterns prove across vendors.