Groundhog Day for AI agents: Prompt caching patterns for using OpenAI models on Amazon Bedrock

The AWS Machine Learning Blog published a walkthrough of prompt caching patterns for running OpenAI models on Amazon Bedrock, aimed at teams building multi-turn agents. The post is organized around an agent's lifecycle: its first turn, later turns that accumulate history and tool results, mid-session tool changes, and subagents spawned as helpers. Its central claim is that caching the static and growing parts of an agent's input cuts input cost substantially; the authors report that on GPT-6 Astra in a six-turn test workload, implicit caching delivered roughly 62 percent lower input cost than running without caching, and explicit rolling breakpoints saved about 61 percent. They note these are workload results and recommend measuring your own agent's savings.
The problem the post describes is structural. A model predicts the next token only from the input it receives and has no memory between requests, so every agent step resends the tool definitions, the developer instructions, and the entire prior history of tool calls and results. In a long coding session, turn 20 re-reads turns 1 through 19, and the caller pays for those tokens each time. Prompt caching addresses this by storing the processed input. During prefill the model converts every input token into key-value vectors, its working state, before emitting a first token; caching saves that state, and a later request whose input begins with the same tokens loads it rather than recomputing, processing only what is new.
Amazon Bedrock offers two modes. Implicit caching lets the service pick where the reusable prefix ends, typically following the conversation as it grows. Explicit caching lets code mark the boundary using prompt_cache_breakpoint. The post recommends combining them: an explicit marker after shared tools and instructions gives every new session and helper agent a reusable starting point, while implicit caching tracks each conversation's expanding history automatically.
In the sample code, requests go through the Amazon Bedrock client's responses.create call with a model such as us.openai.gpt-6-astra, a prompt_cache_key that identifies which requests should share a cache, prompt_cache_options carrying mode and ttl, and tools passed before instructions so the rendered order stays stable. Setting store=False keeps conversation history in the application and does not disable caching. In one explicit-caching test, a tools-and-instructions prefix of about 4,000 tokens wrote 4,120 tokens on the first request and read those same 4,120 tokens on later requests; the task text placed after the boundary was not cached.
Exact text matching is the operative constraint. Instructions and tool lists must serialize identically on every request, MCP servers and other tool delivery mechanisms need stable indexing, and anything volatile such as a date, user name or request ID must sit after the cached block. The reusable prefix must also reach 1,024 tokens to qualify.
For the tool-execution loop, the post shows a helper that maps tool names to Python functions, runs requested calls, appends the model's output (including encrypted reasoning) and each tool result to history, and resends until the model returns a final answer. In implicit mode Bedrock can reuse the unchanged prefix and cache through the last tool result. An explicit-only variant wraps tool results in input_text blocks and marks the final result in each batch, keeping earlier markers in place; usage details expose cached_tokens and cache_write_tokens so teams can verify reads and writes. The post flags two failure patterns: a single static breakpoint in explicit-only mode caches tools and instructions but reprocesses growing history every turn (reported at 0.61 relative cost), while an implicit write that ends at a changing task may not leave a reusable shorter prefix for different tasks.
When the tool list changes, keeping the initial catalog stable matters, since adding, removing or reordering definitions can invalidate the cached prefix. For known tools invoked at different steps, the post suggests sending the same sorted tools and restricting calls with allowed_tools as tool_choice, in auto or required mode, without altering the cached catalog. For tools that appear mid-session, the additional_tools input item is accepted by both Bedrock Runtime and Bedrock Mantle endpoints for GPT-6 Astra, GPT-6.1 Sol and GPT-5.6 Luna, with regional availability listed in Bedrock's supported-models table. One example shows the old prefix reading 2,048 tokens and the new write totaling 256. The additional_tools item itself cannot hold a breakpoint, so explicit-only users mark a later input_text block. Different agent roles can instead use stable tool profiles, each with its own cached prefix.
For subagents, the advice is to warm the shared prefix with one ordinary request, confirm cache activity through usage details, then fan out helpers in parallel, each appending its own task to the same marked shared context. The post cautions that caching with agents can leak or expose tools and system prompts, and that responsible AI governance should cover the caching implementation.
Why it matters: Teams running agentic workloads on Bedrock now have concrete patterns, and reported workload figures, for cutting the repeated-token cost that makes long tool-using sessions expensive. This suggests caching discipline becomes a design concern, not just an optimization: prompt ordering, serialization stability and breakpoint placement all shape the bill. It also means caching behavior is worth instrumenting directly, since the reported savings are workload-specific and depend on how carefully prefixes are preserved.
Based on reporting from the original publisher. Visit the source for full context and later updates.
Publisher excerpt
Every time an agent takes a step, it re-reads its entire context before acting. Here is how prompt caching for OpenAI models on Amazon Bedrock lets your agent remember, cutting input cost and latency without resending the same tools, instructions, and history every turn.