gemma 4 26B A4B it GPU requirements: VRAM and cheapest GPU
gemma 4 26B A4B it has 25.8B parameters, published in BF16. It is a mixture-of-experts model: 8 of its 128 experts work on each token, about 4B parameters, so it generates at roughly the speed of a model that size. Every expert still has to sit in GPU memory, which is why the memory figures below are for the full model. It is released under apache-2.0, which allows commercial use.
- Parameters
- 25.8B
- Smallest footprint (INT4)
- about 16 GB
- Cheapest GPU per hour (FP16)
- $0.50
- Type
- Vision
google/gemma-4-26B-A4B-it
- Parameters 25.8B
- Active per token 4B
- Type Vision
- Task image-text-to-text
- Max context 256K
- Licence apache-2.0
- Downloads 5.3M
- Likes 1.8K
Calculating for gemma 4 26B A4B it
More settings
gemma 4 26B A4B it needs about 58 GB of GPU memory
Inference (vLLM), FP16 / BF16, 8K tokens context, one request at a time.
Where the memory goes
- Weights 52 GB
- KV cache 0.17 GB
- Runtime overhead 6.2 GB
- Total 58 GB
GPU counts keep 10% of each card free for spikes. Plan extra if you expect long prompts or many users at once.
| Rank | Configuration | Per GPU / hr | GPUs needed | Total / hr | Total / month (24/7) | Provider |
|---|---|---|---|---|---|---|
| 1 | 1 × RTX PRO 6000 96GBCheapestBlackwell · GDDR7 | $0.50 | 1 | $0.50 | $365 | RunPod (opens in a new tab)1x RTX PRO 6000 MaxQ |
| 2 | 3 × RTX 3090 24GBAmpere · GDDR6X | $0.26 | 3sold as 4 GPUs; you need 3 | $1.06 | $772 | Vast.ai (opens in a new tab)4x RTX 3090 (median of 5 offers) |
| 3 | 2 × A40 48GBAmpere · GDDR6 | $0.59 | 2 | $1.18 | $861 | RunPod (opens in a new tab)2x A40 |
| 4 | 1 × A100 80GBAmpere · HBM2e | $1.21 | 1 | $1.21 | $884 | Vast.ai (opens in a new tab)1x A100 SXM4 (median of 8 offers) |
| 5 | 2 × A100 40GBAmpere · HBM2 | $0.62 | 2 | $1.24 | $903 | Vast.ai (opens in a new tab)2x A100 SXM4 (median of 3 offers) |
| 6 | 2 × RTX A6000 48GBAmpere · GDDR6 | $0.69 | 2 | $1.38 | $1,007 | RunPod (opens in a new tab)2x RTX A6000 |
| 7 | 2 × L40S 48GBAda Lovelace · GDDR6 | $0.79 | 2 | $1.57 | $1,150 | Vast.ai (opens in a new tab)2x L40S (median of 4 offers) |
| 8 | 1 × GH200 96GBHopper · HBM3 | $1.99 | 1 | $1.99 | $1,453 | Vultr (opens in a new tab)vbm-72c-480gb-gh200-gpu (bare metal) |
| 9 | 3 × RTX 4090 24GBAda Lovelace · GDDR6X | $0.53 | 3sold as 4 GPUs; you need 3 | $2.14 | $1,561 | Vast.ai (opens in a new tab)4x RTX 4090 (median of 14 offers) |
| 10 | 2 × RTX 6000 Ada 48GBAda Lovelace · GDDR6 | $1.09 | 2 | $2.18 | $1,591 | RunPod (opens in a new tab)2x RTX 6000 Ada |
| 11 | 1 × MI300X 192GBCDNA 3 · HBM3 | $2.39 | 1 | $2.39 | $1,745 | RunPod (opens in a new tab)1x MI300X |
| 12 | 3 × L4 24GBAda Lovelace · GDDR6 | $0.69 | 3sold as 4 GPUs; you need 3 | $2.76 | $2,015 | RunPod (opens in a new tab)4x L4 |
| 13 | 1 × H100 80GBHopper · HBM3 | $2.89 | 1 | $2.89 | $2,110 | RunPod (opens in a new tab)1x H100 PCIe |
| 14 | 3 × RTX 5090 32GBBlackwell · GDDR7 | $0.73 | 3sold as 4 GPUs; you need 3 | $2.94 | $2,143 | Vast.ai (opens in a new tab)4x RTX 5090 (median of 35 offers) |
| 15 | 1 × H100 NVL 94GBHopper · HBM3 | $3.19 | 1 | $3.19 | $2,329 | RunPod (opens in a new tab)1x H100 NVL |
| 16 | 1 × H200 141GBHopper · HBM3e | $3.79 | 1 | $3.79 | $2,767 | RunPod (opens in a new tab)1x H200 NVL |
| 17 | 3 × A10 24GBAmpere · GDDR6 | $1.42 | 3sold as 4 GPUs; you need 3 | $5.67 | $4,141 | AWS (opens in a new tab)g5.12xlarge |
| 18 | 5 × V100 16GBVolta · HBM2 | $0.79 | 5sold as 8 GPUs; you need 5 | $6.32 | $4,614 | Lambda (opens in a new tab)8x Tesla V100 |
| 19 | 5 × T4 16GBTuring · GDDR6 | $0.98 | 5sold as 8 GPUs; you need 5 | $7.82 | $5,712 | AWS (opens in a new tab)g4dn.metal |
| 20 | 1 × B200 180GBBlackwell · HBM3e | $7.99 | 1 | $7.99 | $5,833 | RunPod (opens in a new tab)1x B200 |
| 21 | 1 × B300 288GBBlackwell Ultra · HBM3e | $8.99 | 1 | $8.99 | $6,563 | RunPod (opens in a new tab)1x B300 |
| 22 | 1 × GB200 186GBBlackwell · HBM3e | $10.50 | 1sold as 4 GPUs; you need 1 | $42.00 | $30,660 | CoreWeave (opens in a new tab)GB200 NVL72 |
| Provider | Instance | Per GPU / hr | GPUs needed | Total / hr | Total / month (24/7) |
|---|---|---|---|---|---|
| RunPod (opens in a new tab)Cheapest | 1x RTX PRO 6000 MaxQus | $0.50 | 1 × RTX PRO 6000 96GB | about $0.50 | about $365 |
| Vast.ai (opens in a new tab) | 4x RTX 3090 (median of 5 offers)worldwide marketplace | $0.26 | 3 × RTX 3090 24GB sold as 4 GPUs; you need 3 | about $1.06 | about $772 |
| Nebius (opens in a new tab) | 1x RTX PRO 6000 96GBeu/us | $1.80 | 1 × RTX PRO 6000 96GB | about $1.80 | about $1,314 |
| Vultr (opens in a new tab) | vbm-72c-480gb-gh200-gpu (bare metal)us | $1.99 | 1 × GH200 96GB | about $1.99 | about $1,453 |
| Lambda (opens in a new tab) | 2x A6000us | $1.09 | 2 × RTX A6000 48GB | about $2.18 | about $1,591 |
| AWS (opens in a new tab) | g7e.2xlargeus-east-1 | $3.36 | 1 × RTX PRO 6000 96GB | about $3.36 | about $2,455 |
| Azure (opens in a new tab) | NC24ads_A100_v4eastus | $3.67 | 1 × A100 80GB | about $3.67 | about $2,681 |
| Google Cloud (opens in a new tab) | g2-standard-48us-central1 | $1.00 | 3 × L4 24GB sold as 4 GPUs; you need 3 | about $4.00 | about $2,921 |
| CoreWeave (opens in a new tab) | GH200us | $6.50 | 1 × GH200 96GB | about $6.50 | about $4,745 |
Prices collected Nov 13, 2026. Every cost is approximate: on-demand, US dollars, mostly US regions (marketplaces such as Vast.ai are worldwide).
Memory by precision and workload
| Precision | Inference | LoRA fine-tune | QLoRA fine-tune | Full fine-tune |
|---|---|---|---|---|
| FP16 / BF16 | 58 GB | 69 GB | – | 458 GB |
| INT8 | 31 GB | – | – | – |
| INT4 | 16 GB | – | 27 GB | – |
Cheapest GPU by precision
Inference with an 8K context and one request at a time.
FP16 / BF16
about 58 GB · 8K tokens
The weights as published. Nothing lost, but the largest footprint.
1 × RTX PRO 6000 96GB
RunPod (opens in a new tab) · 1x RTX PRO 6000 MaxQ
- Per GPU / hr
- $0.50
- GPUs needed
- 1
- Total / hr
- about $0.50
- per month, 24/7, approx.
- $365
INT8
about 31 GB · 8K tokens
Weights at half the size. Most chat and RAG work shows no visible difference.
1 × RTX PRO 6000 96GB
RunPod (opens in a new tab) · 1x RTX PRO 6000 MaxQ
- Per GPU / hr
- $0.50
- GPUs needed
- 1
- Total / hr
- about $0.50
- per month, 24/7, approx.
- $365
INT4
about 16 GB · 8K tokens
Weights at a quarter of the size. Small drops show up first in maths and code.
1 × RTX 3090 24GB
Vast.ai (opens in a new tab) · 1x RTX 3090 (median of 29 offers)
- Per GPU / hr
- $0.25
- GPUs needed
- 1
- Total / hr
- about $0.25
- per month, 24/7, approx.
- $183
Inference vs fine-tuning
Inference (FP16)
1×
of the memory inference needs
58 GB
1 × RTX PRO 6000 96GB · about $0.50
LoRA fine-tune
1.2×
of the memory inference needs
69 GB
1 × RTX PRO 6000 96GB · about $0.50
QLoRA fine-tune
0.5×
of the memory inference needs
27 GB
2 × V100 16GB · about $0.30
Full fine-tune
7.9×
of the memory inference needs
458 GB
3 × MI300X 192GB · about $9.56
Serving it at FP16 fits on 1 × RTX PRO 6000 96GB (RunPod). LoRA needs 1 × RTX PRO 6000 96GB (RunPod), QLoRA 2 × V100 16GB (Vast.ai) and a full fine-tune 3 × MI300X 192GB (RunPod). Training keeps activations and optimiser state in memory as well as the weights, which is why it so often needs more GPUs than serving the same model.
Will it run on a Mac or a single RTX 4090?
RTX 4090 (24 GB)
Yes, at INT4 with an 8K context: about 16 GB fits in the card’s 24 GB.
Mac with Apple silicon
Yes, with enough memory. At INT4 it needs about 16 GB, so a Mac with 24 GB or more of unified memory (macOS lets the GPU use about three quarters of it). Expect slower generation than on a data-centre GPU.
Similar models
Open models of a similar size.
- DiffusionGemma 26B A4BVision25.8B · about 58 GB FP16 · 647.1K ↓GPU needs
- Gemma4 26B A4B QAT Uncensored HauhauCS Balanced MTPVision25.2B · about 58 GB FP16 · 494.1K ↓GPU needs
- Qwen3.8 27B Coder390 EfficientThink Opus5.5 GPT6Astra Grok4.7 DSV4Pro K3 SFT RLOO MTP DFlash2Vision27.3B · about 63 GB FP16 · 915.7K ↓GPU needs
- ClefVision27.4B · about 62 GB FP16 · 73.5K ↓GPU needs
- Qwen3.8-27BVision27.8B · about 63 GB FP16 · 7M ↓GPU needs
- Qwen3.6 27BVision27.8B · about 63 GB FP16 · 2.4M ↓GPU needs
gemma 4 26B A4B it: common questions
How much VRAM does gemma 4 26B A4B it need?
About 58 GB at FP16, 31 GB at INT8 and 16 GB at INT4, with an 8K context and one request at a time. Longer contexts and more parallel requests add KV cache on top.
What is the cheapest GPU to run gemma 4 26B A4B it?
1 × RTX PRO 6000 96GB (about $0.50 per GPU / hour) on RunPod, at about $0.50 an hour for FP16 / BF16 inference with an 8K context. Prices collected Nov 13, 2026; they change often.
Can I fine-tune gemma 4 26B A4B it on a single GPU?
Yes, with QLoRA: it needs about 27 GB, which fits one 80 GB GPU. LoRA needs about 69 GB and a full fine-tune about 458 GB.
How are these numbers calculated?
Weights are parameters times bytes per parameter (2 at FP16, 1 at INT8, 0.5 at INT4). The KV cache comes from the model’s layers, KV heads, head size and context length. We add about 10% plus 1 GB for the runtime and fill each GPU to 90%. Prices are on-demand list prices in US regions, collected from each provider’s public pricing.
Can gemma 4 26B A4B it run on a consumer GPU like the RTX 4090?
Yes, at INT4 with an 8K context: about 16 GB fits in the card’s 24 GB.
Does gemma 4 26B A4B it fit on a single H100 80GB?
Yes, at FP16 / BF16, INT8, INT4 with an 8K context.
How many GPUs do I need for gemma 4 26B A4B it?
At FP16 with an 8K context: 1 × 80 GB, 2 × 48 GB or 3 × 24 GB GPUs. At INT4 you need about a quarter of the memory for the weights.
How much VRAM does gemma 4 26B A4B it need at INT8?
About 31 GB: 27 GB for the weights, 0.17 GB for an 8K KV cache and 3.7 GB of runtime overhead.
How much does it cost to run gemma 4 26B A4B it 24/7 for a month?
About $365 on the cheapest setup we found (1 × RTX PRO 6000 96GB (about $0.50 per GPU / hour), $0.50 an hour for 730 hours). The cheapest on AWS, Google Cloud or Azure is about $2,455 a month.
FP16 vs INT8 vs INT4: which should I use?
FP16 keeps full quality and needs the most memory (about 58 GB here). INT8 halves the weights with little loss for most chat work. INT4 (AWQ, GPTQ or GGUF Q4) cuts them to a quarter, about 16 GB in total, with a small drop in quality that shows most in maths and code.
Can gemma 4 26B A4B it run on a Mac?
Yes, with enough memory. At INT4 it needs about 16 GB, so a Mac with 24 GB or more of unified memory (macOS lets the GPU use about three quarters of it). Expect slower generation than on a data-centre GPU.
vLLM or llama.cpp?
vLLM is built for serving many requests at once on NVIDIA and AMD GPUs and reserves most of the GPU memory for KV cache. llama.cpp runs GGUF files on almost anything, including Macs and CPUs, and suits one user or a small team. Memory needs are similar for the same precision.
Prices collected Nov 13, 2026. Every cost is approximate: on-demand, US dollars, mostly US regions (marketplaces such as Vast.ai are worldwide).
How we calculate
- Weights. Parameters times bytes per parameter: 2 at FP16, 1 at INT8, 0.5 at INT4, plus about 5% for the quantisation scales at INT8 and INT4.
- KV cache. 2 × layers × KV heads × head size × context × requests × 2 bytes, from the model’s config.json. Where a setting is missing we estimate it and say so.
- Overhead. About 10% plus 1 GB for the runtime, in line with what vLLM reserves.
- Fine-tuning. LoRA keeps 16-bit weights and adds a small adapter, its optimiser and activations. QLoRA does the same over 4-bit weights. A full fine-tune needs about 16 bytes per parameter with mixed-precision Adam (weights, gradients and optimiser states), plus activations.
- GPUs. We fill each GPU to 90% of its memory and stop at 8 per node. Costs are on-demand list prices in US regions; marketplace clouds show the median offer.