Skip to content
← FrontpageAmazon hallmark logoAmazonAnalysis

AWS adds prompt caching for open-weight models on Bedrock

Listen to Briefing
0:00/1:30
Topographic elevation contour card for AWS adds prompt caching for open-weight models on Bedrock
VISUAL ARTWORK:Original editorial artwork. Third-party brand marks used under nominative fair use.

What happened.Amazon Web Services launched prompt caching for open-weight foundation models on Amazon Bedrock, reducing inference latency by up to 85 percent and cutting input costs by 90 percent.

Why it matters.Multi-step reasoning loops repeatedly feed system prompts, schema definitions, and conversation histories into foundation models. Applying server-side prompt caching to open-weight models allows developers to deploy complex tool-calling agents without incurring linear token fee growth.

The technical specifications:

  • Latency reduction: Speeds up response times by up to 85 percent for extended conversational contexts and repetitive agent retrieval prompts.
  • Cost savings: Cuts input token processing charges by 90 percent on cached prefix tokens across supported model checkpoints.
  • Model support: Available immediately for Llama 3.3 70B, Mistral Large 2, and Command R+ endpoints across standard AWS commercial regions.
  • Cache invalidation: Automatically manages cache eviction using a 5-minute time-to-live policy that refreshes dynamically whenever a matching prompt prefix arrives.