AI Provider Pricing: Model Rate Cards and Hidden Fees Compared

The same model often costs a different amount depending on where it's bought. Claude, GPT and Gemini models are each sold through the maker's own API and through one or more cloud platforms, and while the headline rate per token is often identical across channels, the rules around it aren't: which discounts apply, what data residency costs, how tools are billed and whether capacity can be reserved. Those rules shape the bill more than the rate card does.

This page explains how inference pricing works and what each buying channel means for cost, with current rates in the provider guides linked throughout. It sits under our AI cost optimization guide, which covers the rest of the AI bill.

How Providers Price Inference

Every major provider meters language models in tokens and prices them per million, but what a token costs depends on what kind it is and how the request is made. Each of the mechanics below rewards or penalizes a specific way of building an AI application.

Output costs more than input

Output tokens cost three to six times as much as input tokens on the major models, because generating text takes more compute than reading it. That makes response length one of the biggest cost levers an application controls: verbose answers, unconstrained formats and long-form generation drive spend faster than long prompts do.

Reasoning models add a less visible version of the same cost. They generate thinking tokens before answering, those tokens bill as output, and they never appear in the response, so a short answer to a hard question can still produce a large output bill. Most reasoning models let developers set an effort level or token budget, which works as a price control as much as a quality setting.

Cached input rewards stable prompts

Providers discount input they've already processed. A cached prompt prefix typically costs 90% or more below fresh input, and some providers charge a small premium to write to the cache in the first place.

The discount only applies when the beginning of a prompt is identical from one request to the next, so prompt structure matters. System instructions, tool definitions and reference documents placed first and kept stable get cached, while anything that changes per request, like a timestamp or user ID near the top, breaks the cache for everything after it. Agents, chat assistants and retrieval over the same documents resend large, stable context on every call and gain the most.

Long prompts can reprice the whole request

Some models charge a higher rate once a prompt passes a size threshold, usually somewhere between 100,000 and 300,000 tokens, and at several providers the higher rate applies to the entire request rather than only the tokens past the line. A request just over the threshold can cost noticeably more than one just under it. Other models now include their full context window at the standard rate. Where a surcharge applies, trimming conversation history, summarizing long sessions and retrieving only the relevant passages keeps requests below it.

Speed is priced in both directions

Most providers now sell the same model at several speeds, and the cost-efficient pattern is to route each workload by how urgently it needs an answer.

Data residency carries a premium

Keeping processing inside a specific geography usually costs more than letting the provider route requests globally, typically around 10% at Anthropic, AWS and Google for the models they price both ways. For workloads with real residency requirements, that premium is a cost of compliance. For everything else, global endpoints are the cheaper default.

Tools and agents bill outside the rate card

A model's rate card covers tokens, but several charges sit outside it, which is why agentic workloads tend to cost more than a per-token estimate suggests:

• Server-side tools such as web search, file search and code execution charge per use.
• Content those tools retrieve is billed again as input tokens.
• Tool definitions are resent as input on every call.
• Conversation history grows with each turn and is resent each time.
• Agent platforms can add runtime or storage charges.

The full AI token economics guide shows how these pieces add up per request, feature and customer, and the AI pricing guide covers the pricing models behind subscriptions, seats and APIs.

Comparing Model Prices

List prices span roughly a hundredfold range, from around $0.10 per million input tokens for small models to $10 for the top flagships, and within each tier the major providers price close together. That makes the rate card a weak guide to which model is actually cheaper for a given job. Three things break a straight rate comparison:

• Tokenizers differ, so the same text counts as a different number of tokens on different models. Anthropic's newer models, for instance, count about 30% more tokens for the same text than its earlier ones.
• Output length and reasoning vary, so two models with identical rates can generate very different amounts of billable output for the same task.
• Cheaper models fail more often on harder tasks, which adds retries.

The measure that accounts for all three is cost per completed task: the cost of a call divided by how often it succeeds on your own workload. Benchmarks help narrow a shortlist, but only a sample of real requests settles it.

The LLM cost comparison has current rates across providers and worked examples of the calculation, and nOps AI model recommendations apply it to actual usage, pairing the projected savings from a model switch with a quality score.

AWS — Amazon Bedrock

Bedrock's pricing advantage is flexibility. It serves models from Anthropic, Meta, Mistral, DeepSeek, Amazon and others, including OpenAI's open-weight models, behind one API, so moving a workload to a cheaper model is a configuration change rather than a migration. Spend lands on the AWS bill next to the infrastructure around it, and each request can choose its own speed and price through service tiers.

One application can run its user-facing features on Standard, its evaluations and background agents on Flex, and its nightly processing in batch, all against the same models.

Two Bedrock-specific costs are easy to miss. Fine-tuned and custom models must be served on Provisioned Throughput, an hourly commitment, so customization carries a fixed monthly cost beyond the training run. And on older Claude Sonnet models, sessions that grow past about 200K tokens switch to separately metered long-context billing; nOps long-context recommendations find the sessions that trigger it.

Because switching models on Bedrock is cheap, the harder question is which model to switch to. nOps compares current OpenAI and Gemini usage against Bedrock options such as Claude, DeepSeek and GPT OSS, and recommends the lowest-cost model that meets the workload's context, latency and function-calling needs. The Bedrock pricing guide has current rates by model family, and Bedrock cost optimization covers the remaining levers.

Azure — Azure OpenAI & Microsoft Foundry

On Azure, the main pricing decision is where data is allowed to go. Microsoft Foundry (formerly Azure AI Foundry) sells OpenAI models alongside models from Anthropic, xAI, DeepSeek, Meta and others, and every deployment picks a processing scope.

Each step toward tighter residency costs more, so Global is the sensible default unless compliance rules it out. Standard bills per token, Batch costs half with a 24-hour turnaround, and Provisioned reserves capacity in provisioned throughput units (PTUs), covered below.

Several costs sit outside the model itself. Fine-tuned models add a hosting charge for every day they stay deployed, whether or not they receive traffic, so unused fine-tunes are a common source of waste. Retrieval setups add Azure AI Search, storage, private networking and egress on separate meters. And Claude in Foundry is operated by Anthropic and billed through Azure Marketplace rather than as an Azure OpenAI meter, which matters for anyone allocating AI spend by provider.

The Microsoft Foundry pricing guide has current rates, and Azure OpenAI cost optimization covers how to lower them.

Google Cloud — Vertex AI

Google rebranded Vertex AI as Gemini Enterprise Agent Platform in April 2026, though the Vertex AI name is still widely used. Its pricing follows the same pattern as the other clouds, with global endpoints cheapest, regional endpoints about 10% more, Priority and Flex or Batch tiers on either side of Standard, and higher rates above 200K tokens on Pro models. Two mechanics work differently.

The first is caching. Gemini caches repeated prompt content automatically, and teams can also create explicit caches that guarantee the discount, but explicit caches carry an hourly storage fee for as long as they exist. An explicit cache pays off for a large context reused heavily within a short window; one that sits idle costs money without saving any.

The second is grounding. Grounding responses with Google Search comes with a monthly free allowance and then bills per query, but Google doesn't charge for the search content it adds to the prompt, the reverse of how web search tools bill on most other APIs.

Gemini isn't always the cheapest model for a workload already running on Google Cloud. nOps Gemini to Bedrock recommendations flag workloads that would cost less on another model with the same context and latency requirements. For steady, high-volume Gemini traffic, Provisioned Throughput is the main lever, and the Vertex AI pricing guide covers current rates.

Direct API Providers

Buying straight from Anthropic, OpenAI or xAI usually means getting new models and API features first, on the same rate cards the clouds start from. The trade-off is a separate invoice and a separate set of usage data, outside the cloud bill where the rest of the infrastructure is tracked, which makes AI spend harder to allocate unless something brings the two together. Each provider's pricing also has its own emphasis.

Anthropic

Anthropic's newer models charge as little as 2.5% to 5% of the input rate for cache reads, and its larger models include the full 1M-token context window at standard rates, which favors applications built on long, stable context. Claude API pricing has every model's rate, the older Anthropic API pricing guide covers billing basics, and reducing Claude API costs covers caching, batching and routing in practice.

OpenAI

OpenAI has the widest spread of speed tiers, from half-price Flex to Ultrafast at several times the standard rate. It counts reasoning tokens against the same output limit as the visible answer, so a request that runs out of budget mid-reasoning is still billed. OpenAI API pricing lists the full rate card, and reducing OpenAI API costs covers the levers that lower it.

xAI

xAI bills server-side tool calls as their own line item and then charges again for the retrieved content as input, so tool-heavy agents cost more there than token counts alone suggest. Grok API pricing covers each model and tool.

Committing to AI Capacity

All three clouds sell reserved inference capacity, billed whether or not it's used, at a lower effective rate than on-demand. The products differ, but the deciding question is the same: will traffic stay high and steady enough to keep the capacity busy?

Reserved capacity pays off at roughly 70% or higher sustained utilization; below about 50%, the reservation is mostly paying for idle capacity. Traffic above the reservation falls back to on-demand on all three clouds, which makes sizing to the 80th percentile of hourly usage, rather than the peak, the usual starting point. Measuring hourly consumption for a few weeks on pay-as-you-go before committing gives the data to size it.

The same discipline applies to cloud compute commitments, covered in the full guide to GCP commitments, and the Microsoft Foundry pricing guide covers PTU sizing in more detail.

How nOps Helps Optimize AI Costs

Fully optimizing AI spend means understanding how token consumption, model choice, and application demand affect the broader cloud environment powering them — across AWS, GCP, and Azure, and across every model provider running on top of them.

That's where nOps comes in: helping FinOps teams understand, allocate, and optimize AI token costs alongside the rest of their cloud and AI infrastructure.

• Unified visibility: Get all of your spending from AWS, GCP, Azure, AI, and SaaS in one place, with cost allocation by application, customer, team, or business unit to understand what's driving AI spend and where optimization will have the greatest impact.
• Commitment Management: Automatically maximize discounts and minimize commitment risk across eligible cloud infrastructure supporting your AI workloads. Customers typically save ~20% by switching to nOps — and with results-based pricing, you pay only when you get better results.

We've talked to companies that can save millions on their cloud bills by switching to nOps from competitors. Book a free savings analysis to quantify exactly how much more you could save across the infrastructure supporting your AI workloads and the rest of your cloud environment.

nOps manages $5B+ in cloud spend and was recently rated #1 in G2's Cloud Cost Management category.

Demo

AI-Powered Cost Management Platform

Discover how much you can save in just 10 minutes!

Book a Demo

FAQ