Prompt Caching for Device System Prompts: When It Lowers the Bill

Illustration

Caching lowers the bill when the same long prefix is reused inside the cache lifetime. With Anthropic's published multipliers (1.25 times to write, 0.1 times to read, 5-minute default lifetime), the second request in a window already saves money. A prefix used only once pays the 1.25 times write rate for nothing.

Many AI devices send the same long system prompt with every request. Prompt caching lets the provider reuse that fixed prefix at a lower price. Whether it saves money depends on how often the same prefix arrives inside the cache lifetime. This guide gives you the published multipliers, a break-even formula and the measurements to take on your own traffic. It contains no Duobay price and no measured result.

What do the providers publish?

These are the Anthropic caching multipliers, read on October 31, 2026. They apply to the base input price of the model you call.

How the three providers differ

The mechanisms differ, and so does what each vendor tells you. Where a vendor has not published a number on its page, this article leaves it out.

Minimum lengths and model lists change often. Re-read each page for your exact model before you decide. This article gives no dollar prices because base prices vary by model and date. Look yours up on the provider pricing page and record the date.

When does caching start to pay?

Let P be the tokens in the fixed prefix, p the base input price per token, and n the number of requests that reuse the prefix inside one lifetime. The first request writes the cache and the next n − 1 requests read it. Output tokens and the changing part of the prompt are billed at their normal rates.

no_cache_cost   = n × P × p
cached_cost     = (m_w + r × (n − 1)) × P × p
ratio(n)        = (m_w + r × (n − 1)) / n          prefix cost only

with changing tokens s per request (billed at p, uncached):
ratio_total(n)  = (m_w × P + r × P × (n − 1) + n × s) / (n × (P + s))

caching wins when:   m_w + r × (n − 1) < n
                     n > (m_w − r) / (1 − r)

Units: P and s are tokens. p is dollars per token. n is a count of requests. m_w and r are multipliers with no unit. With m_w = 1.25 and r = 0.1, the break-even is n > 1.15 / 0.9, which is about 1.28, so the first whole number that wins is n = 2. With the 1-hour write, m_w = 2, so n > 1.9 / 0.9, about 2.11, so you need n = 3 or more.

Computed from the Anthropic multipliers 1.25 (write) and 0.1 (read), as of October 31, 2026, source Anthropic caching docs. A ratio of 1 means the same cost as no caching.

Worked example with assumed inputs

These inputs are illustrations you should replace. Assume P = 4,000 prefix tokens, s = 500 changing tokens per request, and n = 10 requests inside one lifetime, at the 5-minute write and the 0.1 read multiplier.

• No caching: 10 × (4,000 + 500) = 45,000 input tokens billed at p.
• With caching: 1.25 × 4,000 + 0.1 × 4,000 × 9 + 10 × 500 = 5,000 + 3,600 + 5,000 = 13,600 token equivalents at p.
• Ratio: 13,600 / 45,000 is about 0.302. The prefix alone is 0.215. The changing tokens pull the total ratio up, so a long changing suffix shrinks the benefit.

Multiply the token equivalents by your own p to get dollars. Output tokens are not in this calculation.

How does a fleet set n?

Each use refreshes the lifetime at no extra charge, so what matters is the gap between consecutive uses of the prefix. A request is a read if the gap since the previous use is within the lifetime. A run of uses linked by such gaps is a chain, and n is the length of the chain: the uses between two gaps longer than the lifetime. If the identical prefix reaches the provider from many devices, their requests can keep one entry alive together. Confirm cache scope with the scope test below before you count shared hits.

Evaluate in this order.

1. Decide whether the prefix is shared. If the identical prefix comes from F devices and the scope test passed, the stream is the pooled traffic of the fleet. Otherwise the stream is one device.
2. Take the gap g of that stream. For one device, g is its own gap. For the pooled stream under evenly spaced arrivals, g is about I / F minutes. With real traffic, use the measured pooled 90th percentile of gaps.
3. Apply the gap rule below to g and to the expected chain length n.

g <= 5 minutes       : 5-minute write. Every use after the first is a read,
                       so the ratio approaches r. Needs n >= 2, else skip caching.
5 < g <= 60 minutes  : 1-hour write. It wins once n >= 3,
                       because 2 + r × (n − 1) < n. With the 5-minute write
                       every use misses and costs 1.25 times.
g > 60 minutes       : no caching for this stream.

expected uses per 5-minute window, evenly spaced arrivals:
per-device:   5 / I          (meaningful only when I <= 5)
fleet-shared: F × 5 / I      (F = devices sending the identical prefix)

Every gap value falls into one branch: up to 5, above 5 up to 60, or above 60. Within a branch, the chain length decides between caching and not caching. The two window formulas are expected counts under evenly spaced arrivals. Random arrivals can give fewer, so use the measured gap percentile for the decision when you have it. If gaps are mixed, the rule is fed by the percentile you name in advance, for example the 90th.

Assume F = 60 devices and I = 30 minutes. One device alone has g = 30, so by the per-device branch it takes the 1-hour write if its chain reaches n = 3 or more. With a shared prefix and a passed scope test, the pooled gap is about 30 / 60 = 0.5 minutes, which is in the first branch, so the 5-minute write applies to the shared stream. Its expected count is 60 × 5 / 30 = 10 per 5-minute window, and the ratio from the table is 0.215. Steady fleet traffic keeps the entry alive across many windows, so the ratio approaches r. For stopping device retry storms with a circuit breaker, see Circuit Breaker Pattern for Device Fleets and Cloud Retries.

The 1-hour write also helps a single device with long gaps. Take 6 requests from one device at 10-minute spacing, all inside one hour. With the 5-minute write, every request misses: 6 × 1.25 = 7.5 against 6, a ratio of 1.25. With the 1-hour write, the cost is 2 + 5 × 0.1 = 2.5 against 6, a ratio of about 0.417. If the same device keeps this rhythm, each use refreshes the entry and the chain continues. This example is arithmetic on assumed spacing.

Inputs, and which are yours to replace

What are the common cost traps?

These follow from the published mechanics above. They come from engineering reasoning about how the numbers behave. None describes an incident at any company.

• Per-device text inside the prefix. Cache hits require 100 percent identical segments up to the marked block. If a device ID, a timestamp or a user name sits before the marker, no two devices send the same prefix and nothing is ever read from cache. Put fixed text first and per-device text after it.
• Sparse devices that share nothing. If one device talks every 20 minutes and its prefix is private to it, every request lands after the 5-minute lifetime has ended. Every request then pays the 1.25 times write rate on the prefix, which costs more than no caching. If the device keeps a steady rhythm under an hour, the 1-hour write is the candidate.
• A prefix below the minimum length. Anthropic processes a short prefix without caching and returns no error. A prompt you trimmed during a firmware rewrite can quietly fall under the line, and the bill rises with no alert unless you log cached token counts.
• Using the 1-hour write too early. The 1-hour write costs 2 times the base rate. It beats no caching only when the chain of uses it keeps alive has 3 or more uses: 2 + 0.1 × (n − 1) is below n once n is 3 or more. Refresh extends the chain, so count the uses between two gaps longer than an hour. For a chain of one or two uses, it loses.
• Counting a guaranteed hit. OpenAI states that caching does not guarantee that every request reuses stored data or has lower latency. Model the hit rate as an assumption and run a low case.
• A prompt update that resets the cache. Any edit to the text before the marker changes the prefix. After a prompt change rolled out to the fleet, each distinct prefix pays a new write. Roll prompt changes in waves and budget for the extra writes.
• Applying one provider's numbers to another. Anthropic publishes write and read multipliers. OpenAI publishes a ceiling of 95 percent off cached input. Google publishes neither on its caching page. Do not copy the Anthropic formula to the other two.

What should you measure on your own board and backend?

The published pages do not tell you your request rhythm, your hit rate or your cache scope. We have not measured them, and no result appears in this article. Measure them on a prototype before you build the cost model into a price.

Fix every pass line before you run the test. A threshold chosen after you see the result tends to land just under the result.

What should you ask before you commit?

Take this list to your team, and to any supplier who proposes a cloud design for your device.

• How many tokens are in our fixed prefix, and does it clear the minimum for the exact model we call?
• What is the gap between requests for one device, and for the pooled fleet, at the percentile we named?
• Which fields in the prompt change per device or per request, and are they all placed after the cache marker?
• Does the plan use the 5-minute or the 1-hour write, and which request pattern justifies it?
• What read multiplier does our model list on the provider page, and on what date did we read it?
• What does the unit cost look like with a zero hit rate? Can the product margin absorb that case?
• How do we roll out a new prompt version without every device writing a new cache at once?
• Do we log cached token counts for every response, and who looks at them?
• For OpenAI or Gemini, which numbers come from the invoice and which from the docs?
• Who owns the prefix text, and who approves an edit that resets the cache?

If you want to plan the cloud side of your device with a system architecture and a costed BOM, tell us about your device. You can also read how our engineering gates work.

Sources

All pages accessed October 31, 2026. Multipliers, minimum lengths and model lists change, so re-read them before you decide.

• Anthropic, prompt caching documentation: https://platform.claude.com/docs/en/build-with-claude/prompt-caching (accessed 2026-10-31)
• OpenAI, prompt caching guide: https://developers.openai.com/api/docs/guides/prompt-caching (accessed 2026-10-31)
• Google, Gemini API context caching: https://ai.google.dev/gemini-api/docs/caching (accessed 2026-10-31)