Amazon Bedrock Cost Optimization: Reduce LLM Spend in 2026

In 2026, GenAI cost optimization is becoming a much bigger priority. Bedrock workloads are moving beyond small experiments into production applications, coding assistants, RAG pipelines, and multi-step agents—and the costs are scaling with them.

Model choice is only one part of the equation. Bedrock costs are shaped by input and output tokens, prompt length, caching, inference mode, Provisioned Throughput, and the infrastructure supporting Knowledge Bases, RAG pipelines, and agentic workflows. A workload that looks inexpensive during testing can quickly become a significant production expense once request volume scales.

This guide covers seven practical ways to optimize Amazon Bedrock costs.

1. Understand Why Bedrock Costs Add Up

Bedrock spend comes from more than the headline price per token. Here are the main cost drivers:

Per-token billing that scales with every request

Bedrock charges by the token — input tokens for what you send, output tokens for what the model generates. That sounds simple until you realize a 3,000-token system prompt gets billed on every single call. If you're running 100,000 requests per day, that repeated system prompt alone generates 9 billion input tokens per month.

According to AWS documentation, Claude Sonnet 4.6 charges $3.00 per million input tokens and $15.00 per million output tokens on on-demand pricing. For high-throughput workloads, that can mean five-figure monthly bills before you've even optimized the prompts.

Traffic spikes make this worse. Bedrock spend is notoriously difficult to anticipate because usage can spike or plummet day to day depending on user behavior. You can't forecast token consumption the way you'd forecast EC2 hours — unpredictable input length, variable model outputs, and hidden contextual expansion all add noise.

Idle Provisioned Throughput and always-on Knowledge Base infrastructure

Provisioned Throughput gives you dedicated capacity for predictable latency—but you pay for that capacity whether or not you fully use it. When traffic is variable or model units are oversized, idle throughput can quickly become a major source of waste. Custom models do not automatically require Provisioned Throughput, because Bedrock now supports on-demand deployment for supported custom models. However, workloads that use dedicated capacity still need to be sized carefully to avoid paying for unused model units.

Then there’s the infrastructure around Bedrock. Knowledge Bases can use OpenSearch Serverless and other supported vector stores. OpenSearch Serverless Classic maintains baseline indexing and search capacity even during quiet periods, while NextGen collections can scale both to zero after ten minutes without requests. Storage charges and any configured minimum capacity still remain, so the cost depends heavily on the collection type and capacity settings.

2. Pick the Right Model for the Job

The most capable model is rarely the most economical choice for every request. Matching model capability to task complexity can reduce spend without sacrificing quality.

Tiered routing — cheap models for simple tasks, premium only when needed

Not every request needs Claude Opus. Simple classification, summarization, or FAQ responses can run on cheaper models like Nova Lite or Haiku and deliver nearly identical results. Routing logic should match task complexity to model capability.

Here’s what that could look like in practice: a financial services application routes low-risk customer queries to Claude Haiku or Nova Lite, escalating only when confidence scores drop below a threshold. Premium models like Sonnet or Opus handle the edge cases, while routine requests stay on lower-cost models. The exact savings depend on the traffic mix and model pricing, but this can substantially reduce the blended cost per request.

The challenge is building the routing layer. You need classification logic that can decide in milliseconds which model to invoke — and fallback rules when the cheap model fails. But once it's in place, you're no longer paying Opus prices for trivial tasks.

Intelligent Prompt Routing and model distillation

Amazon Bedrock Intelligent Prompt Routing automatically routes requests between models in the same family based on predicted response quality, helping balance cost and performance without requiring a custom routing layer.

Model distillation takes this a step further. You use a large, expensive model (the "teacher") to generate training data, then fine-tune a smaller, cheaper model (the "student") to mimic the teacher's behavior on your specific use case. The result? A model that's 80% as good for 20% of the cost.

AWS reports that distilled models can be up to 75% less expensive than the original model, with less than 2% accuracy loss for use cases such as RAG. The tradeoff is the upfront cost and effort required to generate training data and fine-tune the student model.

3. Cut Tokens Before You Cut Anything Else

Every unnecessary input or output token is multiplied across your full request volume. Prompt design and caching therefore offer some of the fastest savings available.

Prompt optimization and output token limits

Reducing a 3,000-token system prompt to 1,500 tokens cuts input costs by 50% before any other optimization. That's the simplest lever to pull.

Start by auditing your prompts. Strip out redundant examples, compress instructions, and remove filler language. If you're passing the same context on every request (company policies, product catalogs, style guides), move it to a cached system message instead of repeating it in the user prompt.

Output token limits are just as critical. If you only need a 200-word summary but the model generates 1,000 words by default, set max_tokens to cap the response. You're paying for every token the model outputs — don't leave it open-ended.

Prompt caching (up to 90% off cached tokens)

Amazon Bedrock introduced prompt caching in 2025, and as of January 26th, Claude Haiku 4.5, Sonnet 4.5, and Opus 4.5 now support 1-hour cache TTLs. Here's how it works: you mark the static portion of your prompt as cacheable (the system message, reference documents, long-context examples). Bedrock stores that segment server-side for up to an hour. Subsequent requests that reuse the same cached content get charged as low as 10% of the normal input token rate.

For multi-turn conversations, document Q&A, and agentic workflows where the same context gets reused across dozens of requests, this is a 90% cost reduction on cached tokens. The catch? You need to structure your prompts so the cacheable portion stays consistent across requests — dynamic content goes in the non-cached part.

4. Match Consumption Mode to Workload

Bedrock offers several ways to pay for inference, each suited to different latency and volume requirements. The cheapest option depends on whether the workload is real-time, flexible, or predictable at scale.

On-Demand vs Batch (~50% cheaper) vs Flex tier

On-Demand is great for experimentation and low-volume production workloads. You pay per token, no commitment, instant access. But if you're running high-throughput inference at scale, you're overpaying.

Bedrock Batch Processing costs roughly 50% less than On-Demand — perfect for offline jobs like bulk document processing, data enrichment, or nightly report generation. The tradeoff? Batch jobs can take hours to complete instead of milliseconds. If you don't need real-time responses, batch is a no-brainer.

AWS also introduced a Flex pricing tier for certain models that sits between On-Demand and Provisioned Throughput. It gives you discounted per-token rates in exchange for slightly longer response times (think seconds instead of sub-second). If your use case can tolerate that latency, Flex can cut costs by 20-50% compared to On-Demand.

When Provisioned Throughput actually pays off

Provisioned Throughput makes sense when you have steady, high-volume workloads that need consistent low-latency responses. The breakeven point? When your monthly token spend on On-Demand exceeds what you'd pay for the equivalent hourly Provisioned Throughput capacity.

For example, if you're spending $60,000/month on Claude Sonnet On-Demand tokens, and Provisioned Throughput for the same model costs $50,000/month (hourly rate × 720 hours), you save $10,000 by switching. But if your usage drops 30% next month, you're locked into that $50k commitment and you're underwater.

The safe approach? Start with On-Demand, measure your actual usage patterns for 2-3 months, then model out whether Provisioned Throughput would save money. Don't commit until you have data.

5. Apply Commitments to Steady Bedrock & AI Usage

Bedrock pricing optimization is typically one of the most impactful savings levers. Here are some key considerations:

Sizing Provisioned Throughput model units against real usage

If you decide Provisioned Throughput makes sense, size it conservatively. AWS sells model units — each unit provides a specific throughput capacity (tokens per minute). Buy too many units and you're paying for unused capacity. Buy too few and you're throttled, forcing overflow traffic back to On-Demand at full price.

The best practice is to start with the minimum commitment (often 1 model unit for 1 month), measure utilization, then scale up incrementally. Avoid 6-month commitments until usage is stable.

Savings Plans on the compute under your AI stack — RAG, agents, vector DBs

Here's what most teams miss: Bedrock tokens are just the tip of the iceberg. The real spend is in the infrastructure powering your AI workloads — the Lambda functions orchestrating agents, the EKS clusters running your RAG pipeline, the EC2 instances hosting vector databases like Weaviate, Qdrant or Milvus.

That infrastructure is where nOps Commitment Management comes in. Our system continuously adjusts Savings Plans and Reserved Instances in small increments based on your actual usage patterns — covering EC2, Lambda, and Fargate usage supporting ECS and EKS workloads. You maximize effective savings and minimize lock-in with zero manual effort. For teams running production AI workloads, that typically translates to 40-60% savings on the compute layer.

The key is to treat Bedrock token costs and compute costs as a single optimization problem. Cutting tokens by 30% is great — but if your RAG orchestration layer is running on unoptimized On-Demand EC2, you're missing the bigger opportunity.

6. Don't Ignore the Infrastructure Around Bedrock

Model inference is only one layer of the AI stack. Knowledge Bases, vector stores, orchestration, and persistent compute can become significant cost drivers of their own.

Knowledge Bases, OpenSearch Serverless and vector store spend

Knowledge Bases for Amazon Bedrock use OpenSearch Serverless under the hood (among other supported vector stores). You're paying for OCUs — both indexing OCUs (for ingesting and updating vectors) and search OCUs (for queries). Each OCU costs $0.24/hour, and AWS auto-scales them based on load.

The cost profile differs significantly between Classic and NextGen collections. With OpenSearch Serverless Classic, the first production collection in an account generally requires at least two OCUs—one for indexing and one for search. At $0.24 per OCU-hour, that is approximately $346 per month before storage, even when traffic is low. Additional capacity is added as indexing, data, and query requirements increase.

NextGen collection groups default to no minimum OCU capacity and can scale indexing and search independently to zero after ten minutes without requests. To control costs, use NextGen collections where they meet your requirements, avoid configuring minimum capacity unless consistent baseline performance is necessary, remove stale embeddings, share compute through collection groups, and set maximum OCU limits to prevent unexpected scaling.

Lambda, EKS and EC2 costs behind agents and orchestration

LangChain agents, multi-step RAG pipelines, and agentic workflows all require orchestration logic — usually Lambda functions or EKS pods. Those costs add up fast when you're invoking dozens of Lambda functions per Bedrock request or running a persistent EKS cluster for agent coordination.

Lambda costs are driven primarily by execution time and memory usage. If an agent workflow invokes 10 functions per request, with each function using 2 GB of memory for five seconds, processing 1 million requests per month would cost roughly $1,667 in Lambda compute alone.

EKS clusters under standard support have a flat $0.10/hour control plane fee ($72/month), plus the cost of EC2 worker nodes. A small 3-node cluster running t3.medium instances (On-Demand) costs roughly $75/month in EC2 charges on top of the control plane fee. But switch those nodes to Spot instances, and you're down to $25/month — a 67% reduction.

7. Attribute Gen AI Spend

Optimization starts with knowing who and what is driving usage.

Application inference profiles, tags and their limits

AWS lets you attribute Bedrock costs using application inference profiles. You create a profile for a specific model, tag it by team, product, environment, or workload, and invoke the model through the profile ARN. Once activated as cost allocation tags, those tags flow into AWS Cost Explorer and Cost and Usage Reports.

For more granular attribution, you can attach request metadata to individual InvokeModel or Converse calls and enable model invocation logging. This lets you track dimensions that vary by request, such as customer, feature, or experiment, although the metadata appears in your invocation logs rather than directly in Cost Explorer or CUR.

The main limitation is consistency. Application inference profiles are model-specific, while request metadata is not automatically enforced by Bedrock. To prevent attribution gaps, teams can add metadata through a shared client or LLM gateway rather than relying on every developer to include it manually.

Unit economics for AI (cost per request, per feature, per customer)

Tags get you cost by team or product. Unit economics get you cost per user request, per feature invocation, or per customer. That's the data finance teams actually care about.

Here's an example: a SaaS company with an AI-powered code assistant tracks Bedrock spend per user, per month. They discover that the top 10% of users account for 70% of token spend. Now they can make informed decisions: increase pricing for power users, set per-user rate limits, or optimize prompts specifically for high-frequency workflows.

To calculate unit economics, you need to tie Bedrock invocations back to application-level metadata (user IDs, feature flags, customer accounts). That usually means custom logging in your application layer — capturing Bedrock request IDs, token counts, and business context in a central data warehouse, then joining that against your CUR data.

It's not trivial to set up. But once you have it, you can answer questions like "What's our AI cost per paying customer?" or "Which feature is burning through Bedrock tokens fastest?" — and those answers drive better product and pricing decisions.

The Bottom Line

Tokens are just one piece of your AWS Bedrock costs. The real savings come from matching models to workload complexity, caching aggressively, choosing the right consumption mode, and optimizing the compute infrastructure powering your AI stack.

This is exactly where nOps was designed to help.

nOps Commitment Management handles the compute and infrastructure costs underneath your AI workloads — the EC2 instances, EKS clusters, Lambda invocations, and RDS databases powering your RAG pipelines, agent orchestration, and vector stores. We continuously adjust Savings Plans and Reserved Instances in small increments based on your actual usage. That typically translates to 40-60% savings with zero manual effort.

Our AI Cost & Attribution solution ties Bedrock spend and infrastructure costs together, giving you visibility into which teams, products, and features are driving AI spend — and where to optimize AI costs.

Curious how optimized you are on AI? A 30-minute free savings analysis shows you your current Effective Savings Rate and where the opportunities are. Setup is 5 minutes with no agents or infra changes needed.

nOps manages $4 billion in cloud spend for its customers and is rated 5 stars on G2.

FAQs