Prompt Caching: Mechanics, Provider Pricing, and Cache-Aware Routing

TL;DR

Published September 1, 2026. Provider rates were fetched from each vendor's pricing page on that date; Morph rates come from the same pricing table that bills the API.

Prompt caching is the reuse of a model's KV cache for a prompt prefix it has already processed. A request that begins with the same tokens as an earlier request skips prefill for the matching prefix and bills those tokens at a cached-input rate. Every major provider now sells it: 0.1x base input on Anthropic (0.025x on Claude Fable 5.1), OpenAI GPT-5.6, Gemini 3.x, and Moonshot Kimi K3, 0.032x on DeepSeek V4 off-peak, and $0.26 against $1.25 per 1M on GLM-5.3 744B at Morph. The mechanics are the same everywhere. The write surcharges, minimum prefix sizes, and retention windows are not.

How Prompt Caching Works

A transformer answers a prompt in two phases. Prefill reads every prompt token and computes a key and a value vector per token per layer, the KV cache. Decode then produces output tokens one at a time, attending to that cache. Prefill is compute-bound: its cost grows with prompt length, and for a coding agent the prompt is the entire conversation so far. Decode is memory-bound and produces a few hundred tokens per turn. On the session above, prefill handles 1,252,500 tokens and decode handles 15,000. Prefill is where the money goes.

Prompt caching keeps the KV cache from prefill and reuses it. Three rules define what can be reused:

• Exact prefix match. Attention at position n depends on every token before n. The cache for a token is only valid if all preceding tokens are identical. A request therefore hits from token zero up to the first token that differs from the cached prefix, and never past it.
• Block alignment. Engines store the KV cache in fixed-size blocks and hash whole blocks. Morph caches on roughly 1k-token blocks. A matching prefix that ends mid-block reuses the complete blocks before it; the partial block is prefilled again. Very short prompts have no full block to reuse, which is why every provider publishes a minimum cacheable length.
• Locality. The cache is GPU memory (sometimes tiered to CPU or NVMe) on the worker that ran the prefill. A request only hits if it reaches that worker while the blocks are still resident. Retention is bounded either by a TTL or by LRU eviction under memory pressure.

Multi-turn agent loops are the ideal shape for this. Each turn re-sends the previous turns verbatim and appends one new message, so the entire prompt except the newest turn is a prefix the engine has already processed. The provider reports how much hit in the usage object; on Morph it is usage.prompt_tokens_details.cached_tokens, and the remainder of prompt_tokens bills at the input rate.

For the serving side of the same story, throughput, batching, and where the KV cache lives across GPU, CPU, and NVMe, see LLM inference. For the KV-cache memory techniques that make long prefixes cheap to hold, see grouped-query attention and FP8 quantization.

Prompt Caching Pricing by Provider

Every number below was read from the vendor's own docs on September 1, 2026 (links in Sources). The cache read multiplier is cached input divided by base input. One representative model per vendor keeps the table readable; the same multipliers apply across each vendor's current lineup unless noted.

Two things stand out. First, the read multiplier has converged on 0.1x, with DeepSeek below it and Morph's DeepSeek V4 Flash and GLM-5.3 744B above it in ratio terms but far below it in absolute price: $0.03125 per 1M cached input on morph-dsv4flash is the lowest cached rate in the table after DeepSeek's own off-peak window, and Morph has no peak window (DeepSeek doubles every rate from 01:00 to 04:00 and 06:00 to 10:00 UTC on weekdays). Second, only Anthropic charges to write. A 1.25x write means a prefix must be read at least once more to break even against not caching; at 2x for the 1-hour TTL it must be read twice. Everyone else caches speculatively for free.

Per-vendor detail lives on the model pages: Anthropic API pricing, OpenAI API pricing, Gemini API pricing, DeepSeek API, and Kimi K3 API.

Worked Example: A 30-Turn Coding Agent

The session: a 20,000-token system prompt plus tool definitions, byte-identical every turn. Each turn appends 1,500 tokens of user message, tool results, and assistant reply, and the model writes 500 output tokens. 30 turns. Turn n re-sends the prefix plus n minus 1 turns of history, so prompt tokens across the session sum to 1,252,500. The "90% hits" column bills 1,127,250 of those at the cached rate and 125,250 at the input rate.

90% is conservative for this shape. With exact replay only the first prefill (20,000 tokens) and each later turn's new 1,500 tokens miss: 63,500 of 1,252,500, a 95% hit rate. Anything below 90% on a loop like this means the prefix is changing or the requests are landing on different workers. The usage object on the final turn of an exact-replay session looks like this:

{
  "usage": {
    "prompt_tokens": 63500,
    "completion_tokens": 500,
    "total_tokens": 64000,
    "prompt_tokens_details": { "cached_tokens": 62000 }
  }
}

62,000 of 63,500 prompt tokens bill at $0.03125 per 1M on morph-dsv4flash; the 1,500 new tokens bill at $0.1234375. Output is unaffected: caching never touches the decode side of the bill. If your agent's output share is large (always-on reasoning models like Kimi K3 emit long reasoning traces), caching cuts the input line and leaves the output line alone. For the full set of levers on the rest of the bill, see LLM cost optimization and Claude Code token usage.

Cache-Aware Routing: Why a Naive Router Destroys the Hit Rate

An LLM router classifies a prompt and picks the cheapest model that can handle it. Applied per turn to a multi-turn session, it fights prompt caching in two ways.

• Each model has its own cache. A KV cache is a function of the weights. The prefix GLM-5.3 prefilled is useless to DeepSeek V4 Flash. The first turn on a new model prefills the whole conversation at that model's input rate.
• Within a model, the cache is per worker. A load balancer that spreads a conversation across replicas round-robin turns a 95% replay into a hit rate near 1 divided by the replica count. Session affinity, a stable mapping from conversation to worker, is what makes the automatic cache deliver.

Route once, at the session boundary

The first turn of a session is the one moment a switch is free, because nothing is cached yet. Morph's Model Router classifies difficulty, ambiguity, and domain in one call at $0.005 per request, returns the model to call from a candidate list you pin, and then gets out of the way. Every later turn reuses the decision and carries the same session key, so both the model and the worker stay fixed.

import os
import requests
from openai import OpenAI

MORPH_API_KEY = os.environ["MORPH_API_KEY"]
client = OpenAI(api_key=MORPH_API_KEY, base_url="https://api.morphllm.com/v1")

CANDIDATES = ["deepseek-v4-flash", "glm-5.2"]  # two tiers, not ten
_session_model: dict[str, str] = {}


def model_for_session(session_id: str, first_turn: str) -> str:
    # Route once, at the session boundary, when nothing is cached yet.
    if session_id in _session_model:
        return _session_model[session_id]
    resp = requests.post(
        "https://api.morphllm.com/v1/router/multimodel",
        headers={"Authorization": f"Bearer {MORPH_API_KEY}"},
        json={
            "input": first_turn,
            "allowed_models": CANDIDATES,
            "policy": "cost_efficient",
            "default_model": "glm-5.2",
        },
        timeout=5,
    )
    _session_model[session_id] = resp.json()["model"]
    return _session_model[session_id]


def handle_turn(session_id: str, messages: list[dict]):
    model = model_for_session(session_id, messages[-1]["content"])
    return client.chat.completions.create(
        model=model,
        messages=messages,
        extra_body={"prompt_cache_key": session_id},
    )

Two or three candidates is the right size. Every extra candidate is another cold cache the router can dump a session into. Re-classify only at boundaries you are already paying for: a new session, or a context compaction that rewrote the prefix anyway. Morph's Claude Code proxy ships this as a context lock: past 60k tokens of context it stops classifying and holds the session's model, because at that size prefill dominates any per-token rate difference.

Pin the worker with a session key

Caching on Morph needs no key. The key answers the other question: which worker serves the turn. Send one id per conversation as the prompt_cache_key body field or the x-session-id header, on every turn and on retries of the same turn, and each request routes to the worker that already holds its prefix. OpenCode and the Codex CLI already send prompt_cache_key; OpenRouter forwards x-session-id on behalf of its callers. The key changes placement, never billing. cache_ttl is independent: it controls how long that worker keeps the prefix, sliding, in five tiers from 5m to 24h. Invalid values are rejected with a 400.

from openai import OpenAI

client = OpenAI(
    api_key="YOUR_MORPH_API_KEY",
    base_url="https://api.morphllm.com/v1",
)

# One id per conversation, generated when it starts, reused on every
# turn and on retries of the same turn. It pins placement, not billing.
SESSION_ID = "conv-8f2c1a"

response = client.chat.completions.create(
    model="morph-dsv4flash",
    messages=messages,  # system, tools, full history, newest message last
    extra_body={
        "prompt_cache_key": SESSION_ID,
        "cache_ttl": "1h",  # sliding: every hit refreshes the clock
    },
)

usage = response.usage
cached = usage.prompt_tokens_details.cached_tokens
print(f"{cached}/{usage.prompt_tokens} prompt tokens served from cache")

curl -X POST "https://api.morphllm.com/v1/chat/completions" \
  -H "Authorization: Bearer YOUR_MORPH_API_KEY" \
  -H "x-session-id: conv-8f2c1a" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "morph-glm53-744b",
    "messages": [{"role": "user", "content": "..."}],
    "cache_ttl": "6h"
  }'

Escalate a turn without losing the cheap prefix

Some turns in a cheap session genuinely need the stronger model. The mistake is to move the session. Instead, run the hard turn on the strong model under its own session key, append its answer to the shared history, and continue the next turn on the cheap model. Appending never invalidates a prefix: the cheap model's cached blocks still match through the last turn it served, and only the new assistant message and the next user message prefill. The strong model pays one full prefill, which is the price of the escalation, not a recurring tax. If escalations become frequent, the classifier was wrong about the session; re-route at the next compaction.

CHEAP = "morph-dsv4flash"
STRONG = "morph-glm53-744b"


def handle_turn(session_id: str, messages: list[dict], hard: bool):
    # The cheap model keeps its warm prefix under the session key.
    # A hard turn runs on the strong model under its own key. Its answer
    # is appended to the shared history, so the cheap model's prefix is
    # still an exact match on the next turn: append never breaks a hit.
    model = STRONG if hard else CHEAP
    key = f"{session_id}:{model}"
    reply = client.chat.completions.create(
        model=model,
        messages=messages,
        extra_body={"prompt_cache_key": key, "cache_ttl": "1h"},
    )
    messages.append({"role": "assistant", "content": reply.choices[0].message.content})
    return reply

The same rule governs multi-agent layouts: give each subagent its own session key and its own stable prefix rather than sharing one conversation across models. See multi-agent model routing and, for Claude Code specifically, Claude Code Router.

Rules for a High Hit Rate

Measure, do not assume. Log cached_tokens divided by prompt_tokens per request and alert when a session's ratio drops below the replay expectation. On Anthropic the equivalent fields are cache_read_input_tokens and cache_creation_input_tokens; on DeepSeek, prompt_cache_hit_tokens and prompt_cache_miss_tokens. Claude Code exposes the same ratio in /cost.

Pitfalls

Kimi K3 needs reasoning_content replayed verbatim

Kimi K3 thinks on every turn and is trained to see its own prior reasoning. Moonshot's quickstart requires adding the complete assistant message to the next request, reasoning_content and tool_calls included, not content alone. Harnesses that strip the thinking to save tokens degrade the model and, because the stripped history differs from what the model saw, still miss the cache. Replay it and the prefix grows, which is exactly what $0.29 per 1M cached input on morph-kimik3 (against $2.50 uncached) is for. Details on the Kimi K3 API page.

Tool definitions that drift

Tool schemas usually sit right after the system prompt, so a change there invalidates nearly everything. Common causes: an MCP server that re-registers tools with a new description on reconnect, a framework that dedupes tools through a set and loses order, or a dynamic tool loader that injects a per-turn subset. If tools must vary per turn, place the variable subset after the stable ones.

Streaming retries

A stream that drops mid-generation is usually retried with the same request. That retry is a cache hit only if it reaches the same worker, so send the same session key on the retry. Do not append the partial assistant output to the history before retrying: the truncated message becomes part of the prefix on every later turn and differs from what any provider cached.

Editing history

Rewriting an earlier message, re-ranking retrieved chunks, or moving a summary to the top of the prompt changes the prefix at that point. Everything after it misses. This is expected; caching is prefix-based. Append, and when you must rewrite, do it once at a compaction boundary so the new prefix caches fresh and stays stable.

TTL expiry and eviction

Anthropic's default cache lives 5 minutes; a user who reads a long response and replies six minutes later pays a cache write again. OpenAI holds 30 minutes on GPT-5.6. Morph defaults to LRU with no fixed expiry and lets you set a sliding cache_ttl up to 24h for sessions that go idle between turns. Past the TTL the prefix stops hitting entirely, and re-sending it caches it again at the normal input rate; there is no write surcharge on Morph.

FAQ