gpt-oss-120b GPU requirements: VRAM and cheapest GPU

gpt-oss-120b has 116.8B parameters, published in FP4. It is a mixture-of-experts model: 4 of its 128 experts work on each token, about 5.8B parameters, so it generates at roughly the speed of a model that size. Every expert still has to sit in GPU memory, which is why the memory figures below are for the full model. It is released under apache-2.0, which allows commercial use.

Parameters: 116.8B
Smallest footprint (INT4): about 69 GB
Cheapest GPU per hour (FP16): $0.59
Type: Text

openai/gpt-oss-120b

View on Hugging Face (opens in a new tab)Read our review

Parameters 116.8B
Active per token 5.8B
Type Text
Task text-generation
Max context 128K
Licence apache-2.0
Downloads 2.4M
Likes 5.5K

More settings

Context length: 8K tokens

gpt-oss-120b needs about 258 GB of GPU memory

Inference (vLLM), FP16 / BF16, 8K tokens context, one request at a time.

Where the memory goes

Weights 234 GB
KV cache 0.3 GB
Runtime overhead 24 GB
Total 258 GB

GPU counts keep 10% of each card free for spikes. Plan extra if you expect long prompts or many users at once.

GPUs that fit, cheapest firstEvery GPU type we track that can hold this setup on one node, ranked by total hourly cost
Rank	Configuration	Per GPU / hr	GPUs needed	Total / hr	Total / month (24/7)	Provider
1	6 × A40 48GBCheapestAmpere · GDDR6	$0.59	6sold as 8 GPUs; you need 6	$4.72	$3,446	RunPod (opens in a new tab)8x A40
2	2 × MI300X 192GBCDNA 3 · HBM3	$2.39	2	$4.78	$3,489	RunPod (opens in a new tab)2x MI300X
3	6 × RTX A6000 48GBAmpere · GDDR6	$0.69	6sold as 8 GPUs; you need 6	$5.52	$4,030	RunPod (opens in a new tab)8x RTX A6000
4	3 × RTX PRO 6000 96GBBlackwell · GDDR7	$1.72	3sold as 4 GPUs; you need 3	$6.89	$5,033	Vast.ai (opens in a new tab)4x RTX PRO 6000 S (median of 8 offers)
5	4 × A100 80GBAmpere · HBM2e	$2.09	4	$8.36	$6,103	RunPod (opens in a new tab)4x A100 PCIe
6	6 × L40S 48GBAda Lovelace · GDDR6	$1.09	6sold as 8 GPUs; you need 6	$8.72	$6,366	RunPod (opens in a new tab)8x L40S
7	6 × RTX 6000 Ada 48GBAda Lovelace · GDDR6	$1.09	6sold as 8 GPUs; you need 6	$8.72	$6,366	RunPod (opens in a new tab)8x RTX 6000 Ada
8	1 × B300 288GBBlackwell Ultra · HBM3e	$8.99	1	$8.99	$6,563	RunPod (opens in a new tab)1x B300
9	4 × H100 80GBHopper · HBM3	$2.89	4	$11.56	$8,439	RunPod (opens in a new tab)4x H100 PCIe
10	4 × H100 NVL 94GBHopper · HBM3	$3.19	4	$12.76	$9,315	RunPod (opens in a new tab)4x H100 NVL
11	3 × H200 141GBHopper · HBM3e	$3.79	3sold as 4 GPUs; you need 3	$15.16	$11,067	RunPod (opens in a new tab)4x H200 NVL
12	2 × B200 180GBBlackwell · HBM3e	$7.89	2	$15.78	$11,519	Lambda (opens in a new tab)2x B200 SXM6
13	8 × A100 40GBAmpere · HBM2	$1.99	8	$15.92	$11,622	Lambda (opens in a new tab)8x HGX A100
14	2 × GB200 186GBBlackwell · HBM3e	$10.50	2sold as 4 GPUs; you need 2	$42.00	$30,660	CoreWeave (opens in a new tab)GB200 NVL72
More than 8: needs several nodes: RTX 5090 32GB, RTX 4090 24GB, RTX 3090 24GB, L4 24GB, A10 24GB, T4 16GB, V100 16GB

Cost per cloudCheapest offer that fits, per provider. Approximate, prices collected Nov 13, 2026.
Provider	Instance	Per GPU / hr	GPUs needed	Total / hr	Total / month (24/7)
RunPod (opens in a new tab)Cheapest	8x A40us	$0.59	6 × A40 48GB sold as 8 GPUs; you need 6	about $4.72	about $3,446
Vast.ai (opens in a new tab)	4x RTX PRO 6000 S (median of 8 offers)worldwide marketplace	$1.72	3 × RTX PRO 6000 96GB sold as 4 GPUs; you need 3	about $6.89	about $5,033
Nebius (opens in a new tab)	1x B300 288GBeu/us	$9.50	1 × B300 288GB	about $9.50	about $6,935
Vultr (opens in a new tab)	vbm-48c-1024gb-4-a100-gpu (bare metal)us	$2.40	4 × A100 80GB	about $9.59	about $7,000
Azure (opens in a new tab)	NC96ads_A100_v4eastus	$3.67	4 × A100 80GB	about $14.69	about $10,725
Lambda (opens in a new tab)	2x B200 SXM6us	$7.89	2 × B200 180GB	about $15.78	about $11,519
AWS (opens in a new tab)	g7e.24xlargeus-east-1	$4.14	3 × RTX PRO 6000 96GB sold as 4 GPUs; you need 3	about $16.57	about $12,098
Google Cloud (opens in a new tab)	g4-standard-192us-central1	$4.50	3 × RTX PRO 6000 96GB sold as 4 GPUs; you need 3	about $18.00	about $13,140
CoreWeave (opens in a new tab)	L40Sus	$2.25	6 × L40S 48GB sold as 8 GPUs; you need 6	about $18.00	about $13,140

Prices collected Nov 13, 2026. Every cost is approximate: on-demand, US dollars, mostly US regions (marketplaces such as Vast.ai are worldwide).

Memory by precision and workload

With a 8K-token context and one request at a time. LoRA and full fine-tuning start from 16-bit weights and QLoRA from 4-bit, so the other cells are blank.
Precision	Inference	LoRA fine-tune	QLoRA fine-tune	Full fine-tune
FP16 / BF16	258 GB	302 GB	–	2,060 GB
INT8	136 GB	–	–	–
INT4	69 GB	–	112 GB	–

Cheapest GPU by precision

Inference with an 8K context and one request at a time.

FP16 / BF16

about 258 GB · 8K tokens

The weights as published. Nothing lost, but the largest footprint.

6 × A40 48GB

RunPod (opens in a new tab) · 8x A40

sold as 8 GPUs; you need 6

Per GPU / hr: $0.59
GPUs needed: 6
Total / hr: about $4.72
per month, 24/7, approx.: $3,446

INT8

about 136 GB · 8K tokens

Weights at half the size. Most chat and RAG work shows no visible difference.

4 × A40 48GB

RunPod (opens in a new tab) · 4x A40

Per GPU / hr: $0.59
GPUs needed: 4
Total / hr: about $2.36
per month, 24/7, approx.: $1,723

INT4

about 69 GB · 8K tokens

Weights at a quarter of the size. Small drops show up first in maths and code.

1 × RTX PRO 6000 96GB

RunPod (opens in a new tab) · 1x RTX PRO 6000 MaxQ

Per GPU / hr: $0.50
GPUs needed: 1
Total / hr: about $0.50
per month, 24/7, approx.: $365

Inference vs fine-tuning

Inference (FP16)
1×
of the memory inference needs
258 GB
6 × A40 48GB · about $4.72
LoRA fine-tune
1.2×
of the memory inference needs
302 GB
7 × A40 48GB · about $4.72
QLoRA fine-tune
0.4×
of the memory inference needs
112 GB
3 × A40 48GB · about $2.36
Full fine-tune
8×
of the memory inference needs
2,060 GB
8 × B300 288GB · about $71.92

Serving it at FP16 fits on 6 × A40 48GB (RunPod). LoRA needs 7 × A40 48GB (RunPod), QLoRA 3 × A40 48GB (RunPod) and a full fine-tune 8 × B300 288GB (RunPod). Training keeps activations and optimiser state in memory as well as the weights, which is why it so often needs more GPUs than serving the same model.

Will it run on a Mac or a single RTX 4090?

RTX 4090 (24 GB)

Not on its own. Even at INT4 it needs about 69 GB, more than the card’s 24 GB. Two or more cards, or a bigger data-centre GPU, will do it.

Mac with Apple silicon

Yes, with enough memory. At INT4 it needs about 69 GB, so a Mac with 96 GB or more of unified memory (macOS lets the GPU use about three quarters of it). Expect slower generation than on a data-centre GPU.

Run it yourself or pay per token?

Run it yourself

The cheapest setup we found at FP16 is 6 × A40 48GB on RunPod, about $3,446 a month if it runs all the time. Worth it when traffic is steady or the data must stay on your own machines.

Use a hosted API

Our review of gpt-oss-120b covers where to use it through an API and what that costs. Paying per token usually works out cheaper for light or uneven traffic.

Read our review

Similar models

Open models of a similar size.

Calculate another model

gpt-oss-120b: common questions

How much VRAM does gpt-oss-120b need?

About 258 GB at FP16, 136 GB at INT8 and 69 GB at INT4, with an 8K context and one request at a time. Longer contexts and more parallel requests add KV cache on top.

What is the cheapest GPU to run gpt-oss-120b?

6 × A40 48GB (about $0.59 per GPU / hour; sold as 8 GPUs; you need 6) on RunPod, at about $4.72 an hour for FP16 / BF16 inference with an 8K context. Prices collected Nov 13, 2026; they change often.

Can I fine-tune gpt-oss-120b on a single GPU?

Not on one GPU. Even QLoRA needs about 112 GB, more than an 80 GB GPU holds. LoRA needs about 302 GB and a full fine-tune about 2,060 GB.

How are these numbers calculated?

Weights are parameters times bytes per parameter (2 at FP16, 1 at INT8, 0.5 at INT4). The KV cache comes from the model’s layers, KV heads, head size and context length. We add about 10% plus 1 GB for the runtime and fill each GPU to 90%. Prices are on-demand list prices in US regions, collected from each provider’s public pricing.

Can gpt-oss-120b run on a consumer GPU like the RTX 4090?

Not on its own. Even at INT4 it needs about 69 GB, more than the card’s 24 GB. Two or more cards, or a bigger data-centre GPU, will do it.

Does gpt-oss-120b fit on a single H100 80GB?

Yes, at INT4 with an 8K context.

How many GPUs do I need for gpt-oss-120b?

At FP16 with an 8K context: 4 × 80 GB, 6 × 48 GB or 12 × 24 GB GPUs. At INT4 you need about a quarter of the memory for the weights.

How much VRAM does gpt-oss-120b need at INT8?

About 136 GB: 123 GB for the weights, 0.3 GB for an 8K KV cache and 13 GB of runtime overhead.

How much does it cost to run gpt-oss-120b 24/7 for a month?

About $3,446 on the cheapest setup we found (6 × A40 48GB (about $0.59 per GPU / hour; sold as 8 GPUs; you need 6), $4.72 an hour for 730 hours). The cheapest on AWS, Google Cloud or Azure is about $10,725 a month.

FP16 vs INT8 vs INT4: which should I use?

FP16 keeps full quality and needs the most memory (about 258 GB here). INT8 halves the weights with little loss for most chat work. INT4 (AWQ, GPTQ or GGUF Q4) cuts them to a quarter, about 69 GB in total, with a small drop in quality that shows most in maths and code.

Can gpt-oss-120b run on a Mac?

vLLM or llama.cpp?

vLLM is built for serving many requests at once on NVIDIA and AMD GPUs and reserves most of the GPU memory for KV cache. llama.cpp runs GGUF files on almost anything, including Macs and CPUs, and suits one user or a small team. Memory needs are similar for the same precision.