embeddinggemma 300m GPU requirements: VRAM and cheapest GPU
embeddinggemma 300m has 302.9M parameters, published in FP32. Its licence (gemma) adds terms of its own, so read it before you use the model commercially.
- Parameters
- 302.9M
- Smallest footprint (INT4)
- about 1.2 GB
- Cheapest GPU per hour (FP16)
- $0.16
- Type
- Embeddings
google/embeddinggemma-300m
- Parameters 302.9M
- Type Embeddings
- Task sentence-similarity
- Max context 2K
- Licence gemma
- Downloads 972.2K
- Likes 2.1K
Calculating for embeddinggemma 300m
More settings
embeddinggemma 300m needs about 1.7 GB of GPU memory
Inference (vLLM), FP16 / BF16, 2K tokens context, one request at a time.
Where the memory goes
- Weights 0.61 GB
- Runtime overhead 1.1 GB
- Total 1.7 GB
GPU counts keep 10% of each card free for spikes. Plan extra if you expect long prompts or many users at once.
| Rank | Configuration | Per GPU / hr | GPUs needed | Total / hr | Total / month (24/7) | Provider |
|---|---|---|---|---|---|---|
| 1 | 1 × RTX 3090 24GBCheapestAmpere · GDDR6X | $0.16 | 1 | $0.16 | $117 | Lium (opens in a new tab)1x RTX 3090 Ti |
| 2 | 1 × RTX 4090 24GBAda Lovelace · GDDR6X | $0.30 | 1 | $0.30 | $219 | Lium (opens in a new tab)1x RTX 4090 |
| 3 | 1 × RTX 5090 32GBBlackwell · GDDR7 | $0.40 | 1 | $0.40 | $292 | Lium (opens in a new tab)1x RTX 5090 |
| 4 | 1 × RTX A6000 48GBAmpere · GDDR6 | $0.42 | 1 | $0.42 | $307 | Lium (opens in a new tab)1x RTX A6000 |
| 5 | 1 × L4 24GBAda Lovelace · GDDR6 | $0.45 | 1 | $0.45 | $326 | Vast.ai (opens in a new tab)1x L4 (median of 3 offers) |
| 6 | 1 × RTX PRO 6000 96GBBlackwell · GDDR7 | $0.50 | 1 | $0.50 | $365 | RunPod (opens in a new tab)1x RTX PRO 6000 MaxQ |
| 7 | 1 × A40 48GBAmpere · GDDR6 | $0.52 | 1 | $0.52 | $381 | Vast.ai (opens in a new tab)1x A40 (median of 5 offers) |
| 8 | 1 × T4 16GBTuring · GDDR6 | $0.53 | 1 | $0.53 | $384 | AWS (opens in a new tab)g4dn.xlarge |
| 9 | 1 × RTX 6000 Ada 48GBAda Lovelace · GDDR6 | $0.75 | 1 | $0.75 | $548 | Lium (opens in a new tab)1x RTX 6000 Ada Generation |
| 10 | 1 × L40S 48GBAda Lovelace · GDDR6 | $0.80 | 1 | $0.80 | $585 | Vast.ai (opens in a new tab)1x L40S (median of 7 offers) |
| 11 | 1 × A100 40GBAmpere · HBM2 | $0.80 | 1 | $0.80 | $586 | Vast.ai (opens in a new tab)1x A100 SXM4 (median of 15 offers) |
| 12 | 1 × A10 24GBAmpere · GDDR6 | $1.01 | 1 | $1.01 | $734 | AWS (opens in a new tab)g5.xlarge |
| 13 | 1 × A100 80GBAmpere · HBM2e | $1.12 | 1 | $1.12 | $820 | Vast.ai (opens in a new tab)1x A100 SXM4 (median of 16 offers) |
| 14 | 1 × H100 80GBHopper · HBM3 | $1.39 | 1 | $1.39 | $1,015 | Lium (opens in a new tab)1x H100 80GB HBM3 |
| 15 | 1 × GH200 96GBHopper · HBM3 | $1.99 | 1 | $1.99 | $1,453 | Vultr (opens in a new tab)vbm-72c-480gb-gh200-gpu (bare metal) |
| 16 | 1 × MI300X 192GBCDNA 3 · HBM3 | $2.39 | 1 | $2.39 | $1,745 | RunPod (opens in a new tab)1x MI300X |
| 17 | 1 × H200 141GBHopper · HBM3e | $2.90 | 1 | $2.90 | $2,117 | Lium (opens in a new tab)1x H200 NVL |
| 18 | 1 × V100 16GBVolta · HBM2 | $3.06 | 1 | $3.06 | $2,234 | AWS (opens in a new tab)p3.2xlarge |
| 19 | 1 × H100 NVL 94GBHopper · HBM3 | $3.19 | 1 | $3.19 | $2,329 | RunPod (opens in a new tab)1x H100 NVL |
| 20 | 1 × B200 180GBBlackwell · HBM3e | $5.60 | 1 | $5.60 | $4,088 | Lium (opens in a new tab)1x B200 |
| 21 | 1 × B300 288GBBlackwell Ultra · HBM3e | $8.00 | 1 | $8.00 | $5,840 | Lium (opens in a new tab)1x B300 SXM6 |
| 22 | 1 × GB200 186GBBlackwell · HBM3e | $10.50 | 1sold as 4 GPUs; you need 1 | $42.00 | $30,660 | CoreWeave (opens in a new tab)GB200 NVL72 |
| Provider | Instance | Per GPU / hr | GPUs needed | Total / hr | Total / month (24/7) |
|---|---|---|---|---|---|
| Lium (opens in a new tab)Cheapest | 1x RTX 3090 Tiworldwide marketplace | $0.16 | 1 × RTX 3090 24GB | about $0.16 | about $117 |
| Vast.ai (opens in a new tab) | 1x RTX 3090 (median of 32 offers)worldwide marketplace | $0.26 | 1 × RTX 3090 24GB | about $0.26 | about $186 |
| RunPod (opens in a new tab) | 1x RTX 3090 Tius | $0.46 | 1 × RTX 3090 24GB | about $0.46 | about $336 |
| AWS (opens in a new tab) | g4dn.xlargeus-east-1 | $0.53 | 1 × T4 16GB | about $0.53 | about $384 |
| Azure (opens in a new tab) | NC4as_T4_v3eastus | $0.53 | 1 × T4 16GB | about $0.53 | about $384 |
| Google Cloud (opens in a new tab) | g2-standard-4us-central1 | $0.71 | 1 × L4 24GB | about $0.71 | about $516 |
| Lambda (opens in a new tab) | 1x A6000us | $1.09 | 1 × RTX A6000 48GB | about $1.09 | about $796 |
| Nebius (opens in a new tab) | 1x L40S 48GBeu/us | $1.55 | 1 × L40S 48GB | about $1.55 | about $1,132 |
| Vultr (opens in a new tab) | vcg-l40s-16c-180g-48vramus | $1.67 | 1 × L40S 48GB | about $1.67 | about $1,220 |
| CoreWeave (opens in a new tab) | GH200us | $6.50 | 1 × GH200 96GB | about $6.50 | about $4,745 |
Prices collected Nov 16, 2026. Every cost is approximate: on-demand, US dollars, mostly US regions (marketplaces such as Vast.ai and Lium are worldwide).
Memory by precision and workload
| Precision | Inference | LoRA fine-tune | QLoRA fine-tune | Full fine-tune |
|---|---|---|---|---|
| FP16 / BF16 | 1.7 GB | 1.9 GB | – | 6.5 GB |
| INT8 | 1.3 GB | – | – | – |
| INT4 | 1.2 GB | – | 1.4 GB | – |
Cheapest GPU by precision
Inference with an 8K context and one request at a time.
FP16 / BF16
about 1.7 GB · 2K tokens
The weights as published. Nothing lost, but the largest footprint.
1 × RTX 3090 24GB
Lium (opens in a new tab) · 1x RTX 3090 Ti
- Per GPU / hr
- $0.16
- GPUs needed
- 1
- Total / hr
- about $0.16
- per month, 24/7, approx.
- $117
INT8
about 1.3 GB · 2K tokens
Weights at half the size. Most chat and RAG work shows no visible difference.
1 × RTX 3090 24GB
Lium (opens in a new tab) · 1x RTX 3090 Ti
- Per GPU / hr
- $0.16
- GPUs needed
- 1
- Total / hr
- about $0.16
- per month, 24/7, approx.
- $117
INT4
about 1.2 GB · 2K tokens
Weights at a quarter of the size. Small drops show up first in maths and code.
1 × RTX 3090 24GB
Lium (opens in a new tab) · 1x RTX 3090 Ti
- Per GPU / hr
- $0.16
- GPUs needed
- 1
- Total / hr
- about $0.16
- per month, 24/7, approx.
- $117
Inference vs fine-tuning
Inference (FP16)
1×
of the memory inference needs
1.7 GB
1 × RTX 3090 24GB · about $0.16
LoRA fine-tune
1.1×
of the memory inference needs
1.9 GB
1 × RTX 3090 24GB · about $0.16
QLoRA fine-tune
0.9×
of the memory inference needs
1.4 GB
1 × RTX 3090 24GB · about $0.16
Full fine-tune
3.9×
of the memory inference needs
6.5 GB
1 × RTX 3090 24GB · about $0.16
Serving it at FP16 fits on 1 × RTX 3090 24GB (Lium). LoRA needs 1 × RTX 3090 24GB (Lium), QLoRA 1 × RTX 3090 24GB (Lium) and a full fine-tune 1 × RTX 3090 24GB (Lium). Training keeps activations and optimiser state in memory as well as the weights, which is why it so often needs more GPUs than serving the same model.
Will it run on a Mac or a single RTX 4090?
RTX 4090 (24 GB)
Yes, at FP16 / BF16 with an 8K context: about 1.7 GB fits in the card’s 24 GB.
Mac with Apple silicon
Yes, with enough memory. At INT4 it needs about 1.2 GB, so a Mac with 16 GB or more of unified memory (macOS lets the GPU use about three quarters of it). Expect slower generation than on a data-centre GPU.
Similar models
Open models of a similar size.
- DINOv3Embeddings303.1M · about 1.7 GB FP16 · 702.1K ↓GPU needs
- gte multilingual baseEmbeddings305.4M · about 1.7 GB FP16 · 840.1K ↓GPU needs
- Granite Embedding Multilingual R2 (311M and 97M)Embeddings311.7M · about 1.7 GB FP16 · 71K ↓GPU needs
- ruri v3 310mEmbeddings314.6M · about 1.7 GB FP16 · 572.9K ↓GPU needs
- mxbai embed large v1Embeddings335.1M · about 1.7 GB FP16 · 2.1M ↓GPU needs
- UAE Large V1Embeddings335.1M · about 1.7 GB FP16 · 792.7K ↓GPU needs
embeddinggemma 300m: common questions
How much VRAM does embeddinggemma 300m need?
About 1.7 GB at FP16, 1.3 GB at INT8 and 1.2 GB at INT4, with an 8K context and one request at a time. Longer contexts and more parallel requests add KV cache on top.
What is the cheapest GPU to run embeddinggemma 300m?
1 × RTX 3090 24GB (about $0.16 per GPU / hour) on Lium, at about $0.16 an hour for FP16 / BF16 inference with an 8K context. Prices collected Nov 16, 2026; they change often.
Can I fine-tune embeddinggemma 300m on a single GPU?
Yes, with QLoRA: it needs about 1.4 GB, which fits one 80 GB GPU. LoRA needs about 1.9 GB and a full fine-tune about 6.5 GB.
How are these numbers calculated?
Weights are parameters times bytes per parameter (2 at FP16, 1 at INT8, 0.5 at INT4). The KV cache comes from the model’s layers, KV heads, head size and context length. We add about 10% plus 1 GB for the runtime and fill each GPU to 90%. Prices are on-demand list prices in US regions, collected from each provider’s public pricing.
Can embeddinggemma 300m run on a consumer GPU like the RTX 4090?
Yes, at FP16 / BF16 with an 8K context: about 1.7 GB fits in the card’s 24 GB.
Does embeddinggemma 300m fit on a single H100 80GB?
Yes, at FP16 / BF16, INT8, INT4 with an 8K context.
How many GPUs do I need for embeddinggemma 300m?
At FP16 with an 8K context: 1 × 80 GB, 1 × 48 GB or 1 × 24 GB GPUs. At INT4 you need about a quarter of the memory for the weights.
How much VRAM does embeddinggemma 300m need at INT8?
About 1.3 GB, weights and runtime overhead only: this model encodes its input in one pass, so it keeps no KV cache.
How much does it cost to run embeddinggemma 300m 24/7 for a month?
About $117 on the cheapest setup we found (1 × RTX 3090 24GB (about $0.16 per GPU / hour), $0.16 an hour for 730 hours). The cheapest on AWS, Google Cloud or Azure is about $384 a month.
FP16 vs INT8 vs INT4: which should I use?
FP16 keeps full quality and needs the most memory (about 1.7 GB here). INT8 halves the weights with little loss for most chat work. INT4 (AWQ, GPTQ or GGUF Q4) cuts them to a quarter, about 1.2 GB in total, with a small drop in quality that shows most in maths and code.
Can embeddinggemma 300m run on a Mac?
Yes, with enough memory. At INT4 it needs about 1.2 GB, so a Mac with 16 GB or more of unified memory (macOS lets the GPU use about three quarters of it). Expect slower generation than on a data-centre GPU.
vLLM or llama.cpp?
vLLM is built for serving many requests at once on NVIDIA and AMD GPUs and reserves most of the GPU memory for KV cache. llama.cpp runs GGUF files on almost anything, including Macs and CPUs, and suits one user or a small team. Memory needs are similar for the same precision.
Prices collected Nov 16, 2026. Every cost is approximate: on-demand, US dollars, mostly US regions (marketplaces such as Vast.ai and Lium are worldwide).
How we calculate
- Weights. Parameters times bytes per parameter: 2 at FP16, 1 at INT8, 0.5 at INT4, plus about 5% for the quantisation scales at INT8 and INT4.
- KV cache. 2 × layers × KV heads × head size × context × requests × 2 bytes, from the model’s config.json. Where a setting is missing we estimate it and say so.
- Overhead. About 10% plus 1 GB for the runtime, in line with what vLLM reserves.
- Fine-tuning. LoRA keeps 16-bit weights and adds a small adapter, its optimiser and activations. QLoRA does the same over 4-bit weights. A full fine-tune needs about 16 bytes per parameter with mixed-precision Adam (weights, gradients and optimiser states), plus activations.
- GPUs. We fill each GPU to 90% of its memory and stop at 8 per node. Costs are on-demand list prices in US regions; marketplace clouds show the median offer.