Skip to main content

​
Zero configuration

There is nothing to turn on and nothing to mark up. Send requests the way you already do — no cache parameters, no annotations on message blocks, no special headers. When a request repeats context the API has seen before — a stable system prompt, documents pasted into the conversation, a growing message history — the repeated part is recognized and served from cache. There are also no cache lifetimes to manage and nothing to invalidate. If the repeated context is available in cache you get the discount; if it isn’t, the request simply bills at list. Correctness never depends on it.

​
How it bills

Every request draws from your plan’s included usage in proportion to the model you asked for and the tokens it used. Cached input is the exception, on the models that price it: it counts at that model’s published cached input rate — typically 10–20% of its input rate. There is no write premium — fresh input bills at list whether or not it seeds the cache. Which models price cached input is published, not implied — read pricing_usd_per_mtok.cached_input from GET /v1/models, or the model pages. Where a model publishes no cached rate, repeated context is still recognized and still returns faster, but it counts at the ordinary input rate — we don’t discount what the model doesn’t. Usage reporting is protocol-standard on both dialects either way. On the OpenAI shape, prompt_tokens includes the cached subset, broken out under prompt_tokens_details.cached_tokens:
OpenAI dialect — usage on a cached turn
On the Anthropic shape, input_tokens counts fresh input only and cached input is reported as cache_read_input_tokens:
Anthropic dialect — usage on a cached turn
One more case: an identical repeated request — same body, deterministic sampling (temperature 0 or unset) — may be answered instantly and counts at 10% of the request’s list value.

​
Getting the most from it

Caching keys off repetition, so the discount follows how you shape your prompts:
  • Stable content first. Put the system prompt, documents, and examples at the top; the part that changes each turn — the user’s latest message — last.
  • Keep the prefix byte-stable. A timestamp, a random id, or a reordered field near the top of the prompt makes every turn look new. Append to the conversation; don’t rewrite what came before.
  • Long sessions benefit most. Agent sessions and document Q&A resend a large stable prefix on every turn — exactly the shape the cache rewards.
  • Churn benefits least. Tool-call-heavy loops whose context changes substantially between turns repeat less, so less of their input is served from cache.
The workloads that gain the most are text-heavy: long-document analysis, retrieval-augmented chat, and coding sessions over a large stable context.

​
Where it shows up

Two places. First, the usage block of every response, in the dialect-standard fields above — your existing token accounting keeps working unchanged. Second, your included-usage meter: on a model that prices cached input, a request whose input is mostly cached draws far less from the monthly pool than its raw token count suggests. The dashboard meter at /dashboard/usage reflects the same values the API enforces.
Two things cache less than you might expect. Heavy tool-loop turns with fast-changing context have little that repeats — the discount follows repetition and can’t manufacture it. And on a model that publishes no cached rate there is no discount to apply. Budget every workload at list rates and treat cache savings as upside.