GPU calculator
How much GPU does an open model need?
Pick any open model on Hugging Face. We work out the memory it needs, how many GPUs that means, and roughly what it costs to run on AWS, Google Cloud, Azure and the GPU clouds.
Qwen3 0.6B Qwen/Qwen3-0.6B
Prebuilt pageCalculating for Qwen3 0.6B
More settings
Qwen3 0.6B needs about 3.7 GB of GPU memory
Inference (vLLM), FP16 / BF16, 8K tokens context, one request at a time.
Where the memory goes
- Weights 1.5 GB
- KV cache 0.94 GB
- Runtime overhead 1.2 GB
- Total 3.7 GB
GPU counts keep 10% of each card free for spikes. Plan extra if you expect long prompts or many users at once.
| Rank | Configuration | Per GPU / hr | GPUs needed | Total / hr | Total / month (24/7) | Provider |
|---|---|---|---|---|---|---|
| 1 | 1 × RTX 3090 24GBCheapestAmpere · GDDR6X | $0.25 | 1 | $0.25 | $183 | Vast.ai (opens in a new tab)1x RTX 3090 (median of 29 offers) |
| 2 | 1 × V100 16GBVolta · HBM2 | $0.15 | 1sold as 2 GPUs; you need 1 | $0.30 | $220 | Vast.ai (opens in a new tab)2x Tesla V100 (median of 4 offers) |
| 3 | 1 × RTX 4090 24GBAda Lovelace · GDDR6X | $0.48 | 1 | $0.48 | $354 | Vast.ai (opens in a new tab)1x RTX 4090 (median of 34 offers) |
| 4 | 1 × RTX PRO 6000 96GBBlackwell · GDDR7 | $0.50 | 1 | $0.50 | $365 | RunPod (opens in a new tab)1x RTX PRO 6000 MaxQ |
| 5 | 1 × T4 16GBTuring · GDDR6 | $0.53 | 1 | $0.53 | $384 | AWS (opens in a new tab)g4dn.xlarge |
| 6 | 1 × A40 48GBAmpere · GDDR6 | $0.59 | 1 | $0.59 | $431 | RunPod (opens in a new tab)1x A40 |
| 7 | 1 × RTX 6000 Ada 48GBAda Lovelace · GDDR6 | $0.67 | 1 | $0.67 | $488 | Vast.ai (opens in a new tab)1x RTX 6000Ada (median of 8 offers) |
| 8 | 1 × A100 40GBAmpere · HBM2 | $0.68 | 1 | $0.68 | $497 | Vast.ai (opens in a new tab)1x A100 SXM4 (median of 8 offers) |
| 9 | 1 × RTX A6000 48GBAmpere · GDDR6 | $0.69 | 1 | $0.69 | $504 | RunPod (opens in a new tab)1x RTX A6000 |
| 10 | 1 × L4 24GBAda Lovelace · GDDR6 | $0.69 | 1 | $0.69 | $504 | RunPod (opens in a new tab)1x L4 |
| 11 | 1 × L40S 48GBAda Lovelace · GDDR6 | $0.78 | 1 | $0.78 | $571 | Vast.ai (opens in a new tab)1x L40S (median of 6 offers) |
| 12 | 1 × RTX 5090 32GBBlackwell · GDDR7 | $0.87 | 1 | $0.87 | $634 | Vast.ai (opens in a new tab)1x RTX 5090 (median of 55 offers) |
| 13 | 1 × A10 24GBAmpere · GDDR6 | $1.01 | 1 | $1.01 | $734 | AWS (opens in a new tab)g5.xlarge |
| 14 | 1 × A100 80GBAmpere · HBM2e | $1.21 | 1 | $1.21 | $884 | Vast.ai (opens in a new tab)1x A100 SXM4 (median of 8 offers) |
| 15 | 1 × GH200 96GBHopper · HBM3 | $1.99 | 1 | $1.99 | $1,453 | Vultr (opens in a new tab)vbm-72c-480gb-gh200-gpu (bare metal) |
| 16 | 1 × MI300X 192GBCDNA 3 · HBM3 | $2.39 | 1 | $2.39 | $1,745 | RunPod (opens in a new tab)1x MI300X |
| 17 | 1 × H100 80GBHopper · HBM3 | $2.89 | 1 | $2.89 | $2,110 | RunPod (opens in a new tab)1x H100 PCIe |
| 18 | 1 × H100 NVL 94GBHopper · HBM3 | $3.19 | 1 | $3.19 | $2,329 | RunPod (opens in a new tab)1x H100 NVL |
| 19 | 1 × H200 141GBHopper · HBM3e | $3.79 | 1 | $3.79 | $2,767 | RunPod (opens in a new tab)1x H200 NVL |
| 20 | 1 × B200 180GBBlackwell · HBM3e | $7.99 | 1 | $7.99 | $5,833 | RunPod (opens in a new tab)1x B200 |
| 21 | 1 × B300 288GBBlackwell Ultra · HBM3e | $8.99 | 1 | $8.99 | $6,563 | RunPod (opens in a new tab)1x B300 |
| 22 | 1 × GB200 186GBBlackwell · HBM3e | $10.50 | 1sold as 4 GPUs; you need 1 | $42.00 | $30,660 | CoreWeave (opens in a new tab)GB200 NVL72 |
| Provider | Instance | Per GPU / hr | GPUs needed | Total / hr | Total / month (24/7) |
|---|---|---|---|---|---|
| Vast.ai (opens in a new tab)Cheapest | 1x RTX 3090 (median of 29 offers)worldwide marketplace | $0.25 | 1 × RTX 3090 24GB | about $0.25 | about $183 |
| RunPod (opens in a new tab) | 1x RTX 3090 Tius | $0.46 | 1 × RTX 3090 24GB | about $0.46 | about $336 |
| AWS (opens in a new tab) | g4dn.xlargeus-east-1 | $0.53 | 1 × T4 16GB | about $0.53 | about $384 |
| Azure (opens in a new tab) | NC4as_T4_v3eastus | $0.53 | 1 × T4 16GB | about $0.53 | about $384 |
| Google Cloud (opens in a new tab) | g2-standard-4us-central1 | $0.71 | 1 × L4 24GB | about $0.71 | about $516 |
| Lambda (opens in a new tab) | 1x A6000us | $1.09 | 1 × RTX A6000 48GB | about $1.09 | about $796 |
| Nebius (opens in a new tab) | 1x L40S 48GBeu/us | $1.55 | 1 × L40S 48GB | about $1.55 | about $1,132 |
| Vultr (opens in a new tab) | vcg-l40s-16c-180g-48vramus | $1.67 | 1 × L40S 48GB | about $1.67 | about $1,220 |
| CoreWeave (opens in a new tab) | GH200us | $6.50 | 1 × GH200 96GB | about $6.50 | about $4,745 |
Prices collected Nov 13, 2026. Every cost is approximate: on-demand, US dollars, mostly US regions (marketplaces such as Vast.ai are worldwide).
Similar models
Open models of a similar size.
- Qwen3 0.6BText751.6M · about 3.7 GB FP16 · 993.5K ↓GPU needs
- gpt2 largeText811.8M · about 3 GB FP16 · 1M ↓GPU needs
- BitNet b1.58 2B4TText849.8M · about 3.2 GB FP16 · 9.9K ↓GPU needs
- Qwen3 0.6B BaseText596M · about 3.3 GB FP16 · 572.1K ↓GPU needs
- bloomz 560mText559.2M · about 2.7 GB FP16 · 594.7K ↓GPU needs
- Limite 1B - ViolettoText1B · about 3.7 GB FP16 · 4K ↓GPU needs
Qwen3 0.6B: common questions
How much VRAM does Qwen3 0.6B need?
About 3.7 GB at FP16, 2.9 GB at INT8 and 2.5 GB at INT4, with an 8K context and one request at a time. Longer contexts and more parallel requests add KV cache on top.
What is the cheapest GPU to run Qwen3 0.6B?
1 × RTX 3090 24GB (about $0.25 per GPU / hour) on Vast.ai, at about $0.25 an hour for FP16 / BF16 inference with an 8K context. Prices collected Nov 13, 2026; they change often.
Can I fine-tune Qwen3 0.6B on a single GPU?
Yes, with QLoRA: it needs about 2.5 GB, which fits one 80 GB GPU. LoRA needs about 3.7 GB and a full fine-tune about 15 GB.
How are these numbers calculated?
Weights are parameters times bytes per parameter (2 at FP16, 1 at INT8, 0.5 at INT4). The KV cache comes from the model’s layers, KV heads, head size and context length. We add about 10% plus 1 GB for the runtime and fill each GPU to 90%. Prices are on-demand list prices in US regions, collected from each provider’s public pricing.
Can Qwen3 0.6B run on a consumer GPU like the RTX 4090?
Yes, at FP16 / BF16 with an 8K context: about 3.7 GB fits in the card’s 24 GB.
Does Qwen3 0.6B fit on a single H100 80GB?
Yes, at FP16 / BF16, INT8, INT4 with an 8K context.
How many GPUs do I need for Qwen3 0.6B?
At FP16 with an 8K context: 1 × 80 GB, 1 × 48 GB or 1 × 24 GB GPUs. At INT4 you need about a quarter of the memory for the weights.
How much VRAM does Qwen3 0.6B need at INT8?
About 2.9 GB: 0.79 GB for the weights, 0.94 GB for an 8K KV cache and 1.2 GB of runtime overhead.
How much does it cost to run Qwen3 0.6B 24/7 for a month?
About $183 on the cheapest setup we found (1 × RTX 3090 24GB (about $0.25 per GPU / hour), $0.25 an hour for 730 hours). The cheapest on AWS, Google Cloud or Azure is about $384 a month.
FP16 vs INT8 vs INT4: which should I use?
FP16 keeps full quality and needs the most memory (about 3.7 GB here). INT8 halves the weights with little loss for most chat work. INT4 (AWQ, GPTQ or GGUF Q4) cuts them to a quarter, about 2.5 GB in total, with a small drop in quality that shows most in maths and code.
Can Qwen3 0.6B run on a Mac?
Yes, with enough memory. At INT4 it needs about 2.5 GB, so a Mac with 16 GB or more of unified memory (macOS lets the GPU use about three quarters of it). Expect slower generation than on a data-centre GPU.
vLLM or llama.cpp?
vLLM is built for serving many requests at once on NVIDIA and AMD GPUs and reserves most of the GPU memory for KV cache. llama.cpp runs GGUF files on almost anything, including Macs and CPUs, and suits one user or a small team. Memory needs are similar for the same precision.
Prices collected Nov 13, 2026. Every cost is approximate: on-demand, US dollars, mostly US regions (marketplaces such as Vast.ai are worldwide).
Popular models
Prebuilt pages with the numbers worked out at every precision.
- all-MiniLM-L6-v2Embeddings22.7M · about 1 GB FP16 · 173.7M ↓GPU needs
- Qwen3 0.6BText751.6M · about 3.7 GB FP16 · 19.1M ↓GPU needs
- Qwen3 14BText14.8B · about 35 GB FP16 · 13M ↓GPU needs
- Qwen3 Embedding 0.6BEmbeddings595.8M · about 2.3 GB FP16 · 8.6M ↓GPU needs
- DeepSeek V3.2Text685.4B · about 1,509 GB FP16 · 8.4M ↓GPU needs
- bge large en v1.5Embeddings335.1M · about 1.7 GB FP16 · 7.7M ↓GPU needs
- Qwen3 VL 8B InstructVision8.8B · about 22 GB FP16 · 7.1M ↓GPU needs
- multilingual e5 largeEmbeddings559.9M · about 2.2 GB FP16 · 7.1M ↓GPU needs
- Qwen3.8-27BVision27.8B · about 63 GB FP16 · 7M ↓GPU needs
- Gemma 4Vision31.3B · about 71 GB FP16 · 6.6M ↓GPU needs
- MiniMax H3 (Hailuo 3.0)Video33.1B · about 74 GB FP16 · 6.5M ↓GPU needs
- Qwen2.5 7B InstructText7.6B · about 18 GB FP16 · 6.4M ↓GPU needs
- GLM-5.3-FlashVision321.3B · about 708 GB FP16 · 6.4M ↓GPU needs
- Qwen3.5 9BVision9.7B · about 23 GB FP16 · 6.2M ↓GPU needs
- Qwen3 8BText8.2B · about 20 GB FP16 · 6.2M ↓GPU needs
- Unlimited OCRVision3.3B · about 8.9 GB FP16 · 6M ↓GPU needs
- Qwen2.5 VL 7B InstructVision8.3B · about 20 GB FP16 · 5.8M ↓GPU needs
- gpt-oss-20bText20.9B · about 47 GB FP16 · 5.7M ↓GPU needs
- gemma 4 26B A4B itVision25.8B · about 58 GB FP16 · 5.3M ↓GPU needs
- Qwen2.5 1.5B InstructText1.5B · about 4.7 GB FP16 · 5.1M ↓GPU needs
- Qwen3.5 4BVision4.7B · about 12 GB FP16 · 5.1M ↓GPU needs
- Whisper large-v3-turboSpeech808.9M · about 2.8 GB FP16 · 4.9M ↓GPU needs
- dolphin 2.9.1 yi 1.5 34bText34.4B · about 79 GB FP16 · 4.8M ↓GPU needs
- Qwen3.5 2BVision2.3B · about 6.1 GB FP16 · 4.6M ↓GPU needs
- snowflake arctic embed l v2.0Embeddings567.8M · about 2.2 GB FP16 · 4.5M ↓GPU needs
- Qwen2.5 0.5B InstructText494M · about 2.2 GB FP16 · 4.3M ↓GPU needs
- Llama 3.1 8B InstructText8B · about 20 GB FP16 · 4.2M ↓GPU needs
- Qwen3.6 35B A3BVision36B · about 80 GB FP16 · 4M ↓GPU needs
- OmniVoiceSpeech612.6M · about 2.3 GB FP16 · 3.9M ↓GPU needs
- Qwen3 VL 4B InstructVision4.4B · about 12 GB FP16 · 3.5M ↓GPU needs
- Qwen3 4BText4B · about 11 GB FP16 · 3.2M ↓GPU needs
- Qwen3.5 0.8BVision873.4M · about 3 GB FP16 · 3.2M ↓GPU needs
- Qwen3 32BText32.8B · about 75 GB FP16 · 3.1M ↓GPU needs
- Qwen3.8-Flash-NextVision180B · about 397 GB FP16 · 3.1M ↓GPU needs
- Qwen3 1.7BText2B · about 6.5 GB FP16 · 3M ↓GPU needs
- Qwen3 4B Instruct 2507Text4B · about 11 GB FP16 · 2.6M ↓GPU needs
- gpt-oss-120bText116.8B · about 258 GB FP16 · 2.4M ↓GPU needs
- Qwen3.6 27BVision27.8B · about 63 GB FP16 · 2.4M ↓GPU needs
- Qwen3 Embedding 4BEmbeddings4B · about 9.8 GB FP16 · 2.4M ↓GPU needs
- DeepSeek V4 Flash 0731Text304.2B · about 671 GB FP16 · 2.4M ↓GPU needs
- moondream2Vision1.9B · about 6 GB FP16 · 2.4M ↓GPU needs
- Qwen3-Embedding and Qwen3-RerankerEmbeddings7.6B · about 18 GB FP16 · 2.3M ↓GPU needs
- mxbai embed large v1Embeddings335.1M · about 1.7 GB FP16 · 2.2M ↓GPU needs
- Qwen2.5 3B InstructText3.1B · about 8.1 GB FP16 · 2.2M ↓GPU needs
- SAM 3 (Segment Anything Model 3)Vision859.9M · about 2.9 GB FP16 · 2.1M ↓GPU needs
- bge reranker largeEmbeddings559.9M · about 2.2 GB FP16 · 2M ↓GPU needs
- DeepSeek OCRVision3.3B · about 8.9 GB FP16 · 1.9M ↓GPU needs
- NVIDIA Nemotron 3 Nano 4B BF16Text4B · about 11 GB FP16 · 1.9M ↓GPU needs
- GLM-5.3Text753.3B · about 1,659 GB FP16 · 1.9M ↓GPU needs
- Qwen2.5 Coder 14B InstructText14.8B · about 35 GB FP16 · 1.8M ↓GPU needs
How we calculate
- Weights. Parameters times bytes per parameter: 2 at FP16, 1 at INT8, 0.5 at INT4, plus about 5% for the quantisation scales at INT8 and INT4.
- KV cache. 2 × layers × KV heads × head size × context × requests × 2 bytes, from the model’s config.json. Where a setting is missing we estimate it and say so.
- Overhead. About 10% plus 1 GB for the runtime, in line with what vLLM reserves.
- Fine-tuning. LoRA keeps 16-bit weights and adds a small adapter, its optimiser and activations. QLoRA does the same over 4-bit weights. A full fine-tune needs about 16 bytes per parameter with mixed-precision Adam (weights, gradients and optimiser states), plus activations.
- GPUs. We fill each GPU to 90% of its memory and stop at 8 per node. Costs are on-demand list prices in US regions; marketplace clouds show the median offer.