Cloud GPU Pricing Comparison 2026: AWS vs Azure vs GCP

Cloud GPU pricing varies 3-20x across providers depending on commitment model, billing granularity, and utilization rates. AWS, Azure, and Google Cloud Platform (GCP) dominate enterprise AI infrastructure with H100, A100, and V100 instances priced between $1.95/GPU/hour (spot) and $12.29/GPU/hour (on-demand) as of August 2026.

The critical insight most GPU pricing comparisons miss: sticker price is not real cost. At 5% utilization—common for development and early-stage training workloads—an AWS H100 instance with a $12.29/hour list price costs $245.80 per utilized GPU-hour, because 95% of reserved capacity sits idle. Egress charges ($0.08-$0.12/GB), storage for training data and model checkpoints ($0.02-$0.18/GB/month), and idle time between jobs drive total cost of ownership 30-80% above nominal GPU rates.

This guide compares AWS, Azure, and GCP GPU pricing with per-instance hourly rates, explains when spot makes sense, and documents the hidden costs that push real spending far beyond published rate cards.

Hyperscaler GPU Pricing Comparison Table (August 2026)

Because GPU instances bundle multiple accelerators, comparing full instance prices can obscure the underlying economics. The rates below for AWS vs Azure vs Google Cloud GPUpricing are normalized to per-GPU-hour.

Rates are for U.S. East regions (AWS us-east-1, GCP us-central1, Azure East US). Regional variance ranges 10-25% higher in Asia-Pacific and Europe.

Key takeaways from the cloud GPU pricing comparison chart:

• GCP offers the lowest per-GPU on-demand rates for H100 cloud pricing ($9.00-$11.50/GPU/hr) and A100 ($4.05-$4.80/GPU/hr)
• AWS spot instances provide deepest discounts (84-90% off on-demand for H100 and A100)
• Azure lacks true spot pricing — Low Priority VMs offer shallow discounts (20-30% vs 60-84% on AWS/GCP)
• All three providers offer 50-62% savings on 3-year commitments, but lock you into capacity you may not need

Why Your GPU Bill Is Higher Than the Rate Card Shows

Most GPU pricing discussions stop at per-hour rates. Real-world GPU spend includes four additional cost drivers that push total cost 30-80% above sticker price.

1. Utilization Drives Effective Cost Per GPU-Hour

GPU instances bill for uptime, not utilization. A training job that runs 1 hour per day on an 8-GPU instance reserved 24/7 achieves 4% utilization. At AWS p5.48xlarge on-demand rates ($12.29/GPU/hr × 24 hours × 8 GPUs = $2,359.68/day), that single training hour costs $2,359.68 — an effective rate of $294.96 per utilized GPU-hour.

GPU workloads have utilization patterns that swing from 0% to 100% multiple times per week. If your infrastructure does not match this pattern, you are paying for idle GPUs. Web servers maintain 60-80% steady utilization. GPU training jobs are bursty: checkpoint saving, data loading, and hyperparameter tuning create idle periods that cost the same as active training.

2. Egress Charges Can Exceed GPU Costs for Large Model Checkpoints

All three hyperscalers charge for data transfer out of the cloud (egress). AWS, GCP, and Azure typically charge $0.08-$0.12/GB for inter-region and internet egress.

Cross-region egress compounds costs: Moving training data from AWS S3 in us-west-2 to EC2 GPU instances in us-east-1 costs $0.02/GB for intra-AWS transfers. A 1TB training dataset costs $20.48 to replicate. Multi-region training (for fault tolerance or to access scarce GPU capacity) multiplies egress costs linearly.

GCP has an advantage here — Usage.ai noted that "GCP eliminates cross-provider egress costs by co-locating compute and training data on GCP." If your data lake, training datasets, and inference endpoints all live on GCP, you avoid cross-cloud egress entirely. AWS and Azure charge the same rates whether data leaves the cloud or moves between regions.

3. Storage for Training Data and Checkpoints Adds 5-15% Monthly

Training data, intermediate checkpoints, and model artifacts require persistent storage. Hyperscaler object storage pricing:

• AWS S3 Standard: $0.023/GB/month (first 50TB)
• GCP Cloud Storage Standard: $0.020/GB/month
• Azure Blob Storage Hot tier: $0.018/GB/month

Checkpoint storage compounds over time. For example, a team training multiple LLM variants saves 20 checkpoints per model × 5 models × 100GB per checkpoint = 10TB of checkpoint storage. At $0.023/GB/month (AWS S3), that's $230/month in storage costs that persist long after training completes. Checkpoint retention policies ("delete checkpoints older than 30 days") are rarely enforced, causing storage costs to accumulate.

You might consider SSD-backed block storage for training datasets. Some teams use AWS EBS gp3 or Azure Premium SSD for faster dataset access during training. These cost $0.08-$0.13/GB/month — 4-6x more than object storage — but improve training throughput by 20-40% when dataset fits in memory. Trade-off: spend more on storage to reduce GPU hours, or spend more on GPU hours waiting for slower object storage I/O.

4. Per-Hour vs Per-Second Billing Magnifies Short Job Costs

AWS and GCP bill GPU instances per second (60-second minimum). Azure bills per minute (1-minute minimum). This matters most for short-duration inference jobs and batch workloads that frequently start and stop instances.

Per-second billing can reduce wasted spend when thousands of short jobs run for slightly more than a whole-minute interval, while minute-based billing rounds that usage up. For training workloads that run continuously for hours or days, however, billing granularity has little impact on total GPU cost.

Per-second billing matters for batch inference pipelines that spin up GPU instances, process a queue, then terminate. Spot instance interruptions that occur 15 minutes into an hour — you only pay for 15 minutes on AWS/GCP, but Azure rounds to the nearest minute. For training workloads that run hours or days continuously, billing granularity is irrelevant.

When to Use On-Demand, Spot, or Reserved GPU Instances

GPU pricing models (on-demand, spot, reserved) optimize for different workload characteristics. Choosing the wrong model increases costs 2-5x or causes training failures that waste weeks of engineering time.

On-Demand: Predictable Costs, Zero Interruptions

On-Demand is the highest per-hour rate, with no upfront payment, and you pay only for hours used.

Use on-demand when:

• Training runs are critical and cannot tolerate interruptions. Final model training before production deployment, research deadline experiments, customer demos that must complete on schedule.
• Workload duration is under 24 hours. On-demand avoids commitment lock-in for one-off jobs. Spinning up 8x H100 for a 6-hour fine-tuning job costs $589 on-demand (8 GPUs × 6 hours × $12.29/GPU/hr). A 1-year reservation saves 30% but requires paying for 8,760 hours of capacity you won't use.
• Spot capacity is unavailable in your region. H100 spot availability in AWS us-east-1 fluctuates between 0-60% depending on time of day. When spot is exhausted, on-demand is the only option.

Spot/Preemptible: 60-84% Savings, Interruption Risk

AWS Spot Instances and GCP Preemptible VMs offer 60-84% discounts on GPU instances by reclaiming capacity when hyperscaler demand increases. CloudZero documented that "spot and preemptible pricing can lower [H100 costs] to $1.95-$2.50 per GPU per hour on AWS and GCP" versus $9-$12 on-demand.

Use spot when:

• Training runs support checkpointing. PyTorch, TensorFlow, and JAX checkpoint training state every N steps. When a spot instance terminates, training resumes from the last checkpoint on a new instance. Loss: 10-30 minutes of training progress. Gain: 60-84% cost reduction.
• Workload is fault-tolerant and can retry. Hyperparameter sweeps, data preprocessing, batch inference. If a spot instance terminates mid-job, simply relaunch on another instance.
• You run in regions with deep spot capacity. AWS us-west-2, GCP us-central1, and Azure East US typically have better spot availability than Asia-Pacific or secondary US regions.

AWS spot interruption rates for GPU instances range 5-15% depending on instance type and region. P4d (A100) instances see lower interruption rates than older P3 (V100) because fewer customers compete for newer capacity. GCP preemptible VMs terminate after 24 hours maximum runtime even if capacity is available — plan for at least one forced restart per day.

Note that Azure offers Low Priority VMs, but they provide only 20-30% discounts versus 60-84% on AWS/GCP. Azure's spot pricing is not competitive for cost-sensitive AI training.

Reserved Instances: 50-62% Savings, Capacity Lock-In

1-year and 3-year commitments reduce GPU instance costs 30-62% in exchange for paying upfront or committing to hourly usage.

Use reserved instances when:

• Training workload is continuous and predictable. Foundation model pretraining that runs 24/7 for months. Inference clusters serving production traffic with steady request rates.
• You can accurately forecast capacity needs 12-36 months ahead. Over-provisioning a 3-year commitment wastes money. Under-provisioning forces you to pay on-demand rates for overage.
• Spot interruptions are intolerable, but you want on-demand cost savings. Commitments provide on-demand reliability at 30-50% lower cost.

Break-even calculation: AWS 1-year H100 commitment costs approximately $64,000 ($12.29/GPU/hr × 0.70 discount factor × 8,760 hours × 1 GPU). On-demand costs $107,580. Break-even occurs if you use the reserved capacity 60% of the year (5,256 hours). Below 60% utilization, on-demand with aggressive instance lifecycle management is cheaper.

Specialized GPU Cloud Pricing: When Cheap Beats Enterprise

Neocloud GPU providers (RunPod, Lambda Labs, Vast.ai, Latitude.sh, SynpixCloud) offer H100 access starting at $0.42/hour — 87-97% cheaper than hyperscaler on-demand rates. Why the price gap?

Shared tenancy and consumer-grade SLAs: Neocloud providers aggregate spare capacity from data centers, crypto mining operations, and individual GPU owners. Instances may share physical hosts with other tenants, networking bandwidth is best-effort, and uptime SLAs range 95-98% versus 99.95-99.99% on hyperscalers.

Limited compliance and governance: AWS, Azure, and GCP offer HIPAA, SOC 2, FedRAMP, and ISO 27001 certifications. Most neocloud providers do not. Regulated industries (healthcare, finance, government) cannot use neocloud GPUs for production workloads without violating compliance requirements.

No managed service integrations: Hyperscalers integrate GPUs with managed Kubernetes (EKS, GKE, AKS), serverless inference (SageMaker, Vertex AI, Azure ML), and observability tools (CloudWatch, Stackdriver, Azure Monitor). Neocloud GPUs require manual setup of monitoring, autoscaling, and deployment pipelines.

When to use specialized GPU clouds:

• Cost is the primary constraint and compliance is not required. Academic research, open-source model training, personal projects.
• Workload is fully containerized and portable. If your training pipeline runs in Docker and does not depend on AWS-specific APIs, you can run it anywhere.
• Spot-like pricing with less interruption risk. Some neocloud providers offer H100 at $2-3/hour with lower interruption rates than AWS spot because they don't compete with hyperscaler demand spikes.

When specialized clouds do not work:

• Data residency requirements. GDPR, HIPAA, and data sovereignty regulations require data to remain within specific geographic regions. Neocloud providers rarely offer region-specific guarantees.
• Integration with existing cloud infrastructure. If training data lives in AWS S3, model inference runs on Azure Kubernetes, and observability uses GCP BigQuery, running GPUs on a neocloud provider multiplies egress costs and operational complexity.

How to Calculate Real GPU Cost Per Training Run

Sticker price (per-GPU-hour rate) is the starting point, not the total cost. To calculate real GPU cost per training run:

1. GPU instance cost = (per-GPU-hour rate) × (number of GPUs) × (training duration in hours)

2. Idle time cost = (per-GPU-hour rate) × (number of GPUs) × (idle hours between jobs or during checkpoint saves)

3. Storage cost = (checkpoint size in GB) × (number of checkpoints) × (storage rate per GB per month) × (retention period in months)

4. Egress cost = (data transferred out of cloud in GB) × (egress rate per GB)

5. Total cost = GPU instance cost + idle time cost + storage cost + egress cost

Example: Training a 70B parameter LLM on AWS p5.48xlarge (8x H100, $12.29/GPU/hr on-demand) for 72 hours with 10 intermediate checkpoints:

• GPU instance cost: $12.29/GPU/hr × 8 GPUs × 72 hours = $7,079.04
• Idle time cost: Assume 10% idle time during checkpoint saves and data loading: $7,079.04 × 0.10 = $707.90
• Storage cost: 10 checkpoints × 140GB per checkpoint × $0.023/GB/month × 2 months retention = $64.40
• Egress cost: Transfer final checkpoint to on-prem (140GB × $0.12/GB) = $16.80
• Total cost: $7,079.04 + $707.90 + $64.40 + $16.80 = $7,868.14

With spot instances, GPU instance cost drops to $1.95/GPU/hr × 8 GPUs × 72 hours = $1,123.20. Add 20% overhead for spot interruptions and retries: $1,123.20 × 1.20 = $1,347.84. Total cost with spot: $1,347.84 + $707.90 + $64.40 + $16.80 = $2,136.94 — a 73% reduction.

Note that idle time, storage, and egress add 11% to total cost in this example. For teams that retain hundreds of checkpoints or transfer large datasets frequently, non-GPU costs can reach 30-50% of total spend.

AWS vs Azure vs GCP: Which Cloud for GPU Workloads?

Here’s your quick cheat sheet:

Choose AWS When:

Spot instance diversity is critical. AWS offers the widest selection of spot-available GPU instances (P5, P4d, G5, G6) across the most regions. Spot capacity in us-east-1, us-west-2, and eu-west-1 is deeper than Azure or GCP.

You need inference-optimized instances. AWS Inf2 (Inferentia2) and Trn1 (Trainium) instances provide 2-4x better price-performance than GPU-based inference for transformer models. GCP TPUs and Azure do not offer equivalent custom silicon for inference.

Commitment flexibility matters. AWS Reserved Instances and Savings Plans allow capacity exchanges (trade P4d for P5) and regional flexibility. Azure and GCP commitments lock you into specific instance types and regions.

Choose GCP When:

Lowest on-demand rates are essential. GCP H100 instances cost $9.00-$11.50/GPU/hr versus $12.29/GPU/hr on AWS — a 23-27% savings. For teams that cannot use spot or commitments, GCP minimizes on-demand spend.

BigQuery and Vertex AI integration provides value. If training data lives in BigQuery and inference deploys via Vertex AI, co-locating GPU training on GCP eliminates cross-provider egress and simplifies data pipelines.

TPU access is required. GCP is the only hyperscaler offering TPU v5e and v5p for training. TPUs deliver 2-3x better price-performance than GPUs for models optimized for TPU architecture (Transformers, BERT, GPT).

Choose Azure When:

OpenAI model integration is a priority. Azure offers exclusive access to GPT-4-class models via Azure OpenAI Service with guaranteed capacity and enterprise SLAs. Teams building on OpenAI APIs benefit from Azure's co-located inference and training.

Windows VM and .NET ecosystem lock-in exists. If your organization is heavily invested in Azure Active Directory, Windows Server, and .NET microservices, running GPU training on Azure simplifies identity management and networking.

3-year commitments provide the deepest discounts. Azure 3-year reserved instances offer 62% savings versus 50% on AWS and 55% on GCP. If you can commit to 3 years of fixed capacity, Azure delivers the lowest long-term cost.

Optimizing GPU Costs with nOps

Most FinOps teams track GPU spend fragmented across multiple views and sources. This takes manual effort and makes it difficult to allocate costs, calculate unit economics, and report on costs.

nOps was built to solve this with:

Unified visibility Get all of your spending from Azure, AWS, GCP and AI tools all in one place, with cost allocation by model, application, customer, or business unit to understand what is driving spend and where optimization will have the greatest impact.

Commitment Management for more savings and less risk. Customers typically save ~20% on AI workloads by switching to nOps — and with results-based pricing, you pay only when you get better results.

We’ve talked to companies that can save millions on their cloud bills by switching to nOps from competitors. Book a free savings analysis to quantify exactly how much more you can save for no work on your part.

nOps manages $4B+ in cloud spend and was recently rated #1 in G2’s Cloud Cost Management category.

Frequently Asked Questions

Here are a few FAQ about Azure, Google and AWS GPU pricing.