What Is Prompt Caching? How It Cuts AI Costs

Are you sending AI the same long instructions and reference material with every request, and paying full price each time? When the unchanged opening of your prompt is processed from scratch on every call, both your bill and the wait for the first response grow with it.

Prompt caching is a feature that stores the computation for the part of your input that stays the same and reuses it on the next request. Based on OpenAI's official documentation, this post covers how it works, the difference between implicit and explicit caching, the break-even math, and the common mistakes that break the cache.

What is prompt caching? Not recomputing the same opening

Prompt caching processes the repeated opening of your input (the prefix) once, stores it, and reuses that work on later requests. Only the part that changed is processed again, which cuts cost and waiting time.

Say you review dozens of documents against the same rubric. Each input looks like "rubric and examples, then this document." The rubric is the prefix; only the document changes. Three points from OpenAI's documentation matter most:

• The whole prefix must match: reuse only happens when the prefix is identical from start to finish.
• It needs a minimum length: on GPT-5.6 and later, caching starts at 1,024 tokens. Tokens are the units AI models use to measure text.
• It doesn't store answers: only the computation is reused, and the response is generated fresh every time. Caching can shorten processing before the response starts, but a cache hit doesn't guarantee lower latency.

Implicit vs. explicit caching

With implicit caching, OpenAI decides where the reusable section ends. With explicit caching, the developer marks the boundary (a breakpoint) that says "everything up to here repeats."

On GPT-5.6 and later, you can keep the cache for 30 minutes (the default) or 12 hours (Responses API only).

The math: how many reuses before caching pays off?

Caching isn't free. On GPT-5.6 and later, writing to the cache costs more than normal input, and only reads are cheaper, so you need to estimate how often you'll reuse it.

Per the official docs, a cache write costs 1.25x the regular input rate for 30-minute retention, or 2.5x for 12 hours. These are total rates, not surcharges. Cache reads cost 0.1x on most of these models and 0.05x on GPT-6.1 Sol and GPT-6.1 Terra. Earlier models have no extra write charge and use a model-specific cached-input rate. If processing the shared prefix once normally costs 1, and reads cost 0.1x:

The 30-minute cache pays off from the second request, the 12-hour cache from the third. In dollars: on the official pricing page (checked November 4, 2026), GPT-6.1 Terra input is $1.00 per million tokens and cached reads are $0.05. Sending a 20,000-token shared instruction 100 times costs about $2.00 without caching. Assuming the requests keep coming while the cache is alive, it's about $0.12 with caching: a $0.025 write plus about $0.099 in reads. That excludes the per-request questions and output.

Four common mistakes that break the cache

If even one character in the prefix changes, nothing after that point can be reused. The basic rule: put what stays the same first and what changes last.

1. Putting today's date or the current time up front: starting your instructions with "Today is November 4" creates a new cache every day. Move dates and request IDs to the end.
2. Putting user details before shared instructions: add names and personal settings after the shared part.
3. Reordering tools or messages: the same content in a different order is a different prefix.
4. Rewriting earlier turns: append new messages instead. Summarizing a long conversation and feeding it back in can reset reuse, so weigh the smaller input against the lost cache.

To confirm the cache is actually hitting, check cached_tokens (tokens read from cache) and cache_write_tokens (tokens written to cache) in the API usage data.

When you have many tools: tool search and caching

If an agent has dozens or hundreds of tools, including every tool description in every request bloats the input. Tool search loads the detailed definitions of only the tools the model needs, when it needs them.

OpenAI's docs say tool search is designed to preserve the cache by adding newly found tools at the end of the context. But changing the set of tools already loaded breaks the cache from that point on. If you only have a few tools, or most tasks need all of them, loading everything up front is simpler.

Save the instructions you repeat, once

Prepare what never changes once, and process only what's new. The same idea works in everyday AI use.

In daglo, you can save summary instructions such as "decisions, action items, next meeting" in My Templates. daglo remembers the format and summarizes every recording that way as soon as it ends, so the only thing that changes is the meeting itself. Turn on Speaker ID as well, and daglo remembers the voices of the people in your meetings and fills in their real names instead of "Speaker 1" from the next recording on. daglo is the first in Korea to offer it to every user. Learn more about Speaker ID

Read next: for how to measure AI cost per task, caching included, see Cut AI Costs: Measure Cost per Successful Task, Not Tokens.

#PromptCaching #AICostReduction #OpenAIAPI #TokenSaving #ToolSearch #LLMCost #AIProductivity #daglo

👉 Transcribe your next meeting free