Nemotron 3 Ultra GPU requirements: VRAM and cheapest GPU
Nemotron 3 Ultra has 560.5B parameters, published in BF16. It is a mixture-of-experts model: 22 of its 512 experts work on each token, about 55B parameters, so it generates at roughly the speed of a model that size. Every expert still has to sit in GPU memory, which is why the memory figures below are for the full model. Its licence (openmdw-1.1) adds terms of its own, so read it before you use the model commercially.
- Parameters
- 560.5B
- Smallest footprint (INT4)
- about 325 GB
- Cheapest GPU per hour (FP16)
- $2.39
- Type
- Text
nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16
- Parameters 560.5B
- Active per token 55B
- Type Text
- Task text-generation
- Max context 256K
- Licence openmdw-1.1
- Downloads 1M
- Likes 388
Calculating for Nemotron 3 Ultra
More settings
Nemotron 3 Ultra needs about 1,235 GB of GPU memory
Inference (vLLM), FP16 / BF16, 8K tokens context, one request at a time. Some of the model settings were missing, so the KV cache is an estimate.
Where the memory goes
- Weights 1,121 GB
- KV cache 0.54 GB
- Runtime overhead 113 GB
- Total 1,235 GB
GPU counts keep 10% of each card free for spikes. Plan extra if you expect long prompts or many users at once.
| Rank | Configuration | Per GPU / hr | GPUs needed | Total / hr | Total / month (24/7) | Provider |
|---|---|---|---|---|---|---|
| 1 | 8 × MI300X 192GBCheapestCDNA 3 · HBM3 | $2.39 | 8 | $19.12 | $13,958 | RunPod (opens in a new tab)8x MI300X |
| 2 | 8 × B200 180GBBlackwell · HBM3e | $7.69 | 8 | $61.52 | $44,910 | Lambda (opens in a new tab)8x HGX B200 |
| 3 | 5 × B300 288GBBlackwell Ultra · HBM3e | $8.99 | 5sold as 8 GPUs; you need 5 | $71.92 | $52,502 | RunPod (opens in a new tab)8x B300 |
| More than 8: needs several nodes: H200 141GB, GH200 96GB, H100 NVL 94GB, RTX PRO 6000 96GB, H100 80GB, A100 80GB, L40S 48GB, RTX 6000 Ada 48GB, RTX A6000 48GB, A40 48GB, A100 40GB, RTX 5090 32GB, RTX 4090 24GB, RTX 3090 24GB, L4 24GB, A10 24GB, T4 16GB, V100 16GB | ||||||
| Provider | Instance | Per GPU / hr | GPUs needed | Total / hr | Total / month (24/7) |
|---|---|---|---|---|---|
| RunPod (opens in a new tab)Cheapest | 8x MI300Xus | $2.39 | 8 × MI300X 192GB | about $19.12 | about $13,958 |
| Vultr (opens in a new tab) | vbm-256c-2048gb-8-mi300x-gpu (bare metal)us | $3.99 | 8 × MI300X 192GB | about $31.92 | about $23,302 |
| Lambda (opens in a new tab) | 8x HGX B200us | $7.69 | 8 × B200 180GB | about $61.52 | about $44,910 |
| Nebius (opens in a new tab) | 8x B200 180GBeu/us | $8.50 | 8 × B200 180GB | about $68.00 | about $49,640 |
| CoreWeave (opens in a new tab) | HGX B200us | $8.60 | 8 × B200 180GB | about $68.80 | about $50,224 |
| AWS (opens in a new tab) | p6-b200.48xlargeus-east-1 | $14.24 | 8 × B200 180GB | about $113.93 | about $83,171 |
Prices collected Nov 16, 2026. Every cost is approximate: on-demand, US dollars, mostly US regions (marketplaces such as Vast.ai and Lium are worldwide).
Memory by precision and workload
| Precision | Inference | LoRA fine-tune | QLoRA fine-tune | Full fine-tune |
|---|---|---|---|---|
| FP16 / BF16 | 1,235 GB | 1,443 GB | – | 9,878 GB |
| INT8 | 649 GB | – | – | – |
| INT4 | 325 GB | – | 534 GB | – |
Cheapest GPU by precision
Inference with an 8K context and one request at a time.
FP16 / BF16
about 1,235 GB · 8K tokens
The weights as published. Nothing lost, but the largest footprint.
8 × MI300X 192GB
RunPod (opens in a new tab) · 8x MI300X
- Per GPU / hr
- $2.39
- GPUs needed
- 8
- Total / hr
- about $19.12
- per month, 24/7, approx.
- $13,958
INT8
about 649 GB · 8K tokens
Weights at half the size. Most chat and RAG work shows no visible difference.
4 × MI300X 192GB
RunPod (opens in a new tab) · 4x MI300X
- Per GPU / hr
- $2.39
- GPUs needed
- 4
- Total / hr
- about $9.56
- per month, 24/7, approx.
- $6,979
INT4
about 325 GB · 8K tokens
Weights at a quarter of the size. Small drops show up first in maths and code.
8 × A40 48GB
RunPod (opens in a new tab) · 8x A40
- Per GPU / hr
- $0.59
- GPUs needed
- 8
- Total / hr
- about $4.72
- per month, 24/7, approx.
- $3,446
Inference vs fine-tuning
Inference (FP16)
1×
of the memory inference needs
1,235 GB
8 × MI300X 192GB · about $19.12
LoRA fine-tune
1.2×
of the memory inference needs
1,443 GB
6 × B300 288GB · about $71.92
QLoRA fine-tune
0.4×
of the memory inference needs
534 GB
4 × MI300X 192GB · about $9.56
Full fine-tune
8×
of the memory inference needs
9,878 GB
more than one 8-GPU node
Serving it at FP16 fits on 8 × MI300X 192GB (RunPod). LoRA needs 6 × B300 288GB (RunPod), QLoRA 4 × MI300X 192GB (RunPod) and a full fine-tune more than one 8-GPU node. Training keeps activations and optimiser state in memory as well as the weights, which is why it so often needs more GPUs than serving the same model.
Will it run on a Mac or a single RTX 4090?
RTX 4090 (24 GB)
Not on its own. Even at INT4 it needs about 325 GB, more than the card’s 24 GB. Two or more cards, or a bigger data-centre GPU, will do it.
Mac with Apple silicon
Yes, with enough memory. At INT4 it needs about 325 GB, so a Mac with 512 GB or more of unified memory (macOS lets the GPU use about three quarters of it). Expect slower generation than on a data-centre GPU.
Run it yourself or pay per token?
Run it yourself
The cheapest setup we found at FP16 is 8 × MI300X 192GB on RunPod, about $13,958 a month if it runs all the time. Worth it when traffic is steady or the data must stay on your own machines.
Use a hosted API
Our review of Nemotron 3 Ultra covers where to use it through an API and what that costs. Paying per token usually works out cheaper for light or uneven traffic.
Read our reviewSimilar models
Open models of a similar size.
- Ling-3.1-flashText572.1B · about 1,260 GB FP16 · 6.2K ↓GPU needs
- BeamText501.3B · about 1,104 GB FP16 · 13.1K ↓GPU needs
- DeepSeek R1Text684.5B · about 1,508 GB FP16 · 1.6M ↓GPU needs
- DeepSeek V3Text684.5B · about 1,508 GB FP16 · 1.4M ↓GPU needs
- DeepSeek V3.2Text685.4B · about 1,509 GB FP16 · 10.4M ↓GPU needs
- K-EXAONE 2.0Text749.4B · about 1,650 GB FP16 · 14.3K ↓GPU needs
Nemotron 3 Ultra: common questions
How much VRAM does Nemotron 3 Ultra need?
About 1,235 GB at FP16, 649 GB at INT8 and 325 GB at INT4, with an 8K context and one request at a time. Longer contexts and more parallel requests add KV cache on top.
What is the cheapest GPU to run Nemotron 3 Ultra?
8 × MI300X 192GB (about $2.39 per GPU / hour) on RunPod, at about $19.12 an hour for FP16 / BF16 inference with an 8K context. Prices collected Nov 16, 2026; they change often.
Can I fine-tune Nemotron 3 Ultra on a single GPU?
Not on one GPU. Even QLoRA needs about 534 GB, more than an 80 GB GPU holds. LoRA needs about 1,443 GB and a full fine-tune about 9,878 GB.
How are these numbers calculated?
Weights are parameters times bytes per parameter (2 at FP16, 1 at INT8, 0.5 at INT4). The KV cache comes from the model’s layers, KV heads, head size and context length. We add about 10% plus 1 GB for the runtime and fill each GPU to 90%. Prices are on-demand list prices in US regions, collected from each provider’s public pricing.
Can Nemotron 3 Ultra run on a consumer GPU like the RTX 4090?
Not on its own. Even at INT4 it needs about 325 GB, more than the card’s 24 GB. Two or more cards, or a bigger data-centre GPU, will do it.
Does Nemotron 3 Ultra fit on a single H100 80GB?
No. Even at INT4 it needs about 325 GB, so you need several GPUs.
How many GPUs do I need for Nemotron 3 Ultra?
At FP16 with an 8K context: 18 × 80 GB, 29 × 48 GB or 58 × 24 GB GPUs. At INT4 you need about a quarter of the memory for the weights.
How much VRAM does Nemotron 3 Ultra need at INT8?
About 649 GB: 589 GB for the weights, 0.54 GB for an 8K KV cache and 60 GB of runtime overhead.
How much does it cost to run Nemotron 3 Ultra 24/7 for a month?
About $13,958 on the cheapest setup we found (8 × MI300X 192GB (about $2.39 per GPU / hour), $19.12 an hour for 730 hours). The cheapest on AWS, Google Cloud or Azure is about $83,171 a month.
FP16 vs INT8 vs INT4: which should I use?
FP16 keeps full quality and needs the most memory (about 1,235 GB here). INT8 halves the weights with little loss for most chat work. INT4 (AWQ, GPTQ or GGUF Q4) cuts them to a quarter, about 325 GB in total, with a small drop in quality that shows most in maths and code.
Can Nemotron 3 Ultra run on a Mac?
Yes, with enough memory. At INT4 it needs about 325 GB, so a Mac with 512 GB or more of unified memory (macOS lets the GPU use about three quarters of it). Expect slower generation than on a data-centre GPU.
vLLM or llama.cpp?
vLLM is built for serving many requests at once on NVIDIA and AMD GPUs and reserves most of the GPU memory for KV cache. llama.cpp runs GGUF files on almost anything, including Macs and CPUs, and suits one user or a small team. Memory needs are similar for the same precision.
Prices collected Nov 16, 2026. Every cost is approximate: on-demand, US dollars, mostly US regions (marketplaces such as Vast.ai and Lium are worldwide).
How we calculate
- Weights. Parameters times bytes per parameter: 2 at FP16, 1 at INT8, 0.5 at INT4, plus about 5% for the quantisation scales at INT8 and INT4.
- KV cache. 2 × layers × KV heads × head size × context × requests × 2 bytes, from the model’s config.json. Where a setting is missing we estimate it and say so.
- Overhead. About 10% plus 1 GB for the runtime, in line with what vLLM reserves.
- Fine-tuning. LoRA keeps 16-bit weights and adds a small adapter, its optimiser and activations. QLoRA does the same over 4-bit weights. A full fine-tune needs about 16 bytes per parameter with mixed-precision Adam (weights, gradients and optimiser states), plus activations.
- GPUs. We fill each GPU to 90% of its memory and stop at 8 per node. Costs are on-demand list prices in US regions; marketplace clouds show the median offer.