Switches the Anthropic provider from hand-stamping `cache_control` onto
the last message content block over to Anthropic's top-level automatic
prompt caching, paired with explicit long-TTL (1h) anchors on the last
tool definition and on the system prompt.
The prefix order `tools` → `system` → `messages` satisfies Anthropic's
requirement that longer TTLs appear earlier in the prefix, so the static
prefix is cached for 1h (surviving idle gaps longer than the 5-minute
default) while the rapidly-changing conversation tail uses the
free-to-refresh 5-minute TTL via the top-level automatic breakpoint.
Three of the four available cache breakpoints are used (last tool,
system, automatic conversation), leaving one in reserve.
As a side benefit, this fixes a latent issue where the previous stamping
loop could place `cache_control` on a `Thinking` content block, which
the Anthropic API does not allow. Automatic caching is documented to
walk past ineligible blocks (including thinking) when selecting its
breakpoint, so we now delegate that responsibility to the server.
The new shape we send (when caching is enabled):
```json
{
"tools": [{ "...": "...", "cache_control": {"type": "ephemeral", "ttl": "1h"} }],
"system": [
{"type": "text", "text": "...", "cache_control": {"type": "ephemeral", "ttl": "1h"}}
],
"messages": [ /* no per-block cache_control */ ],
"cache_control": {"type": "ephemeral"}
}
```
Release Notes:
- Improved Anthropic prompt cache utilization, reducing latency and cost
for ongoing conversations
---------
Co-authored-by: Martin Ye <martinye022@gmail.com>3 files changedLine totals unavailable