Prompt Caching and Cost

Prompt caching can reduce model cost and latency when an agent repeatedly sends the same system prompt, tool definitions, and conversation history. The model provider owns the cache and its pricing; SyntheticBrew keeps the reusable prefix stable and sends the information supported by the configured provider.

Use this feature for agents with long prompts, large tool catalogs, or several reasoning steps per turn. Short prompts may remain below the provider’s caching threshold and see no benefit.

Understand the two provider behaviors

Automatic caching: the provider recognizes a repeated prefix without an explicit marker. No SyntheticBrew setting is required.
Explicit caching: the provider expects cache_control markers. SyntheticBrew adds those markers for anthropic and openai_compatible model types unless you explicitly disable them.

An OpenAI-compatible gateway may ignore the markers, honor them, or reject them. Check that gateway’s provider documentation before relying on the feature.

Configure explicit caching

For anthropic and openai_compatible, an omitted cache_control setting uses the defaults: enabled, the stable system/tool prefix and conversation history selected, and a minimum estimated prefix of 1,024 tokens.

Use the model REST API when you need to change those defaults:

curl -X PATCH "$SYNTHETICBREW_URL/api/v1/models/claude-haiku" \
  -H "Authorization: Bearer $SYNTHETICBREW_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "cache_control": {
      "enabled": true,
      "breakpoints": ["system", "tools", "history"],
      "min_prefix_tokens": 1024
    }
  }'

breakpoints accepts system, tools, and history. Leave it empty to use the default placement. Increase min_prefix_tokens if your provider charges for cache writes below the point where reads become economical.

To disable explicit markers for a gateway that rejects them:

curl -X PATCH "$SYNTHETICBREW_URL/api/v1/models/custom-gateway" \
  -H "Authorization: Bearer $SYNTHETICBREW_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"cache_control":{"enabled":false}}'

The current Admin model form does not expose cache-control fields. Config import also does not save them. Use the model REST API or a brewctl Model resource:

apiVersion: syntheticbrew/v1
kind: Model
name: claude-haiku
type: anthropic
model_name: claude-haiku-4-5
cache_control:
  enabled: true
  breakpoints: [system, tools, history]
  min_prefix_tokens: 1024

When cache_control is omitted from a brewctl resource, brewctl leaves the model’s saved setting unchanged.

Keep OpenRouter requests cache-friendly

SyntheticBrew sends a stable session identifier with model requests. OpenRouter can use it to keep a conversation on a consistent upstream provider, improving the chance of a warm cache.

Avoid setting provider.order in a model’s extra_body when you want this behavior; OpenRouter treats that setting as an explicit routing choice. If you need to restrict providers, prefer a supported provider.only configuration and test cache results with the exact route.

Verify the result

Run several turns in the same session with a prompt large enough for the provider’s minimum. When the provider reports a cache hit:

the final done SSE event includes cached_prompt_tokens;
the Admin test chat shows the cached-token count in its context bar;
the provider dashboard may show cache writes and cache reads.

cached_prompt_tokens is part of prompt_tokens, not an additional token count. It is omitted when the provider reports no cached tokens.

Expected result: the first request is usually a cache write or cold request, while later requests with the same stable prefix report cached tokens. Exact savings and expiration behavior depend on the model provider.

Troubleshooting

Cached tokens stay at zero: confirm that the provider supports caching, the prompt exceeds its minimum size, and repeated requests use the same session. Also check that explicit caching is not disabled.

The gateway rejects the request: set cache_control.enabled to false. Some OpenAI-compatible gateways reject content-level cache markers.

Cache hits are inconsistent on OpenRouter: remove provider.order, keep the same session, and verify the selected upstream’s cache support and pricing.

The setting changed after an apply: config import does not carry cache_control. Manage it through REST or include it in the brewctl Model resource used by your deployment workflow.