Every major model API now discounts input it has seen recently. A long system prompt, a tool list or a document you send on every turn can be read back from cache for a fraction of the normal input rate. The discount is real. The bookkeeping is where most cost dashboards go wrong.
Reads, writes and plain input
A request's input now comes in up to three kinds, each billed at its own rate:
- Uncached input: billed at the model's normal input rate.
- Cache reads: a prefix served from cache, billed at a steep discount.
- Cache writes: a prefix stored for later, billed at a premium on providers that charge for writes.
| Provider | Cache read | Cache write | Lifetime |
|---|---|---|---|
| OpenAI, GPT-5.6 and later | 0.1× input (0.05× on GPT-6.1 Sol) | 1.25× (30 min) or 2.5× (12 h, Responses API) | 30 min minimum by default |
| OpenAI, earlier models | Model-specific cached-input rate | No write charge | In memory or up to 24 h |
| Anthropic | 0.1× input (lower on some current models) | 1.25× (5 min) or 2× (1 h) | 5 min, refreshed on each hit |
Multipliers are relative to each model's own input price, from the OpenAI and Anthropic caching guides as of October 2026. Check the guide for your model before you trust a number.
The counting trap: "input tokens" means two different things
This is the bug that hides in most home-grown cost code. The two providers disagree on whether the headline input count includes cached tokens.
| OpenAI | Anthropic | |
|---|---|---|
| Headline field | input_tokens / prompt_tokens | input_tokens |
| Includes cached tokens? | Yes: reads and writes are part of it | No: only tokens after the last cache breakpoint |
| Cache read field | input_tokens_details.cached_tokens | cache_read_input_tokens |
| Cache write field | input_tokens_details.cache_write_tokens (newer models) | cache_creation_input_tokens |
| Uncached input | input_tokens − cached − cache_write | input_tokens |
| Total input | input_tokens | input_tokens + cache_read + cache_creation |
Reuse one formula for both and you either bill OpenAI's cached tokens twice (once at full price, once discounted) or drop Anthropic's cached tokens entirely. Both errors scale with exactly the customers who use your product most, because they're the ones with long, cache-friendly conversations.
A worked example
Take a fictional model at $1.00 per million input tokens and an 8,000-token system prompt sent on ten requests within the cache lifetime. Using OpenAI's 30-minute rates (1.25× write, 0.1× read):
| Requests (fictional) | No caching | With caching |
|---|---|---|
| 1st request (write) | $0.0080 | $0.0100 |
| Requests 2–10 (reads) | $0.0720 | $0.0072 |
| Total for the prefix | $0.0800 | $0.0172 |
That's 2.15× the single-request cost instead of 10×, the same ratio OpenAI's guide works through. The first request is more expensive than without caching. If a prefix is only ever used once, a paid write is a small loss.
Why your prompt didn't cache
- It's too short. OpenAI's minimum is 1,024 tokens on GPT-5.6 and later. Anthropic's ranges from 512 to 4,096 depending on the model, and a too-short prompt is silently not cached.
- Something changed near the top. Matching is by exact prefix. A timestamp, user name or reordered tool list at the start invalidates everything after it. Put stable instructions and reference material first and per-request content last.
- The cache expired. Entries live for minutes by default. Low-traffic features may rarely hit.
- Requests landed on different caches. On older OpenAI models, a stable
prompt_cache_keyhelps route related requests together, but it doesn't guarantee a hit.
Measure it rather than guess: cache hit rate is cached tokens divided by total input tokens. Track it per customer as well as per day, because a single customer with a constantly changing prefix can erase the savings you see on average.
Pricing cached tokens per customer
To get cost per customer right, each usage event needs its cache counts and a cached-input rate that was in effect at the event's timestamp. If either is missing, the honest answer is "unknown", not "free" and not "full price".
That's how AgentMargin's price versions behave. Each version carries input, output and an optional cached-input rate in integer microdollars per million tokens. A missing cached rate makes events with cached tokens unknown instead of pricing them at $0. The v0.3 price version has no separate cache-write rate, so for models that charge for writes, compare totals against your provider invoice.
{
"model": "example-model",
"effectiveFrom": "2026-10-22T00:00:00.000Z",
"inputMicrosPerMillion": "1000000",
"outputMicrosPerMillion": "4000000",
"cachedInputMicrosPerMillion": "100000",
"sourceNote": "Synthetic test rate; NOT a provider price quote"
}
