Prompt caching
Prompt caching is an API feature that stores the processed form of a repeated prompt prefix, such as a long system prompt or document, so later requests that start with the same prefix are cheaper and faster.
Also called: cached input tokens, context caching
By Divya Sharma · Checked Nov 16, 2026 · How we review
Many applications send the same opening text again and again: instructions, tool definitions, a large document, or the earlier turns of a chat. Normally the model processes all of it on every request. With prompt caching, the provider keeps the computed state for that prefix for a short time, and a later request with an identical prefix reuses it. Anthropic's documentation describes this as letting the API resume from specific prefixes in your prompts.
How providers do it
Details differ. On the Claude API you mark cache points with `cache_control` or turn on automatic caching; cache entries last 5 minutes by default, with a 1-hour option at extra cost. Cache writes cost more than normal input and cache reads cost much less: by default 0.1 times the base input price, with lower multipliers for some models. OpenAI caches matching prefixes automatically once a prompt is long enough; its documentation gives a minimum of 1,024 tokens for GPT-5.6 and later. Google calls the feature context caching.
How to get cache hits
Only an exact prefix match counts. Put stable content first and changing content, such as the user's question, last. A timestamp or a reordered tool list at the top of the prompt breaks the cache for everything after it. Check the usage fields in responses to confirm cached tokens are being counted.
Not the same as other caches
The KV cache is the in-memory state inside one generation; prompt caching keeps a version of it across requests. A semantic cache is different again: your own system returns a stored answer for a similar question without calling the model. For cost effects, see token pricing.
Related terms
- Token pricing
- Token pricing is how most AI model APIs charge: a set price per million input tokens you send and a separate, usually higher, price per million output tokens the model generates, with discounts for cached input and batch jobs.
- KV cache
- A KV cache stores the attention keys and values a transformer has already computed for earlier tokens, so each new token only needs work for itself. It speeds up generation but takes GPU memory that grows with context length.
- Semantic cache
- A semantic cache stores past LLM responses and returns one when a new request means roughly the same thing as an earlier one, judged by embedding similarity rather than an exact text match. It saves cost and latency on repeated questions.
- Context engineering
- Context engineering is the practice of choosing and maintaining everything that goes into a language model's context window during a task: instructions, retrieved documents, tool definitions, tool results and conversation history.
- Time to first token
- Time to first token (TTFT) is how long a user waits between sending a request to a language model and receiving the first token of its reply, covering queuing, prompt processing and network time.