What is Nemotron 3.5 Bolt (74B-A9B 24GB and E4B 4GB)?
Nemotron 3.5 Bolt is NVIDIA's family of open models for running agents on local hardware. The larger member, Bolt 74B-A9B 24GB, appeared on Hugging Face on 15 October 2026 alongside Nemotron 3.5 Super VL. It is Super VL compressed: an architecture search cut the 120B parent to 74 billion parameters with 9 billion active per token, routing to 9 experts instead of 22, before retraining on 120 billion tokens and reinforcement learning for tool use. Its experts are stored at 2.0625 bits per weight, so the GGUF is 22.02 GB plus a 1.73 GB vision file. It reads text, images and mp4 video up to two minutes and has a 256K-token context.
The smaller member, Bolt E4B 4GB, followed on 23 October. It is not derived from Super VL at all: NVIDIA post-trained Google's Gemma-4-E4B-it with reinforcement learning for multi-turn tool use, instruction following and short spoken replies. It has 8 billion parameters (4 billion effective), takes text only, and has 64K tokens in and out. Its Q4_K_M file is 6.18 GB on disk; NVIDIA's 4GB refers to peak memory in its target workloads, which it measured at 3.1 to 5.0 GB.
Both are under NVIDIA's OpenMDW-1.1 licence, which allows commercial use; E4B is also bound by Gemma 4's Apache 2.0 terms.
Is Nemotron 3.5 Bolt (74B-A9B 24GB and E4B 4GB) worth it?
Bolt 74B-A9B carries the family, and it has the independent number that most local models lack. Artificial Analysis scores it 21.4 on its Intelligence Index, fourth among models under 100B parameters and 1.5 points behind Nemotron 3 Ultra, a 550B model. NVIDIA's own table is more flattering: 50.98 on Terminal-Bench 2.1 against 44.38 for Qwen3.6-35B-A3B, and 66.75 on BrowseComp against 48.74. It is not ahead everywhere even there: Gemma 4 26B-A4B beats it on AA-Omniscience accuracy (22.17 to 16.90) and Qwen edges it on PinchBench.
The name oversells the hardware. The 22 GB file and 1.73 GB vision encoder leave no room for cache on a 24 GB card, and NVIDIA's own deployment table asks for a 32 GB discrete GPU with a 65,536-token context, or 48 GB for the full 262,144 tokens. Supported GPUs are Ada Lovelace and Blackwell only, and every speed figure comes from an RTX Spark laptop with 128 GB of memory.
It is also slower and wordier than its rivals. NVIDIA's blog reports 84 tokens a second with the speculative drafter, against 96.63 for Qwen3.6-35B-A3B and 100.56 for Gemma 4, and 600.80 seconds on an agent task the others finish in about 454 and 481. Artificial Analysis counted 82,000 output tokens per index task, 1.8 times Super VL. Its advantage is accuracy at a memory size, not speed.
E4B is a different bet: a Gemma 4 fine-tune for voice and device assistants. NVIDIA reports 84.79 on IHEval, an instruction-hierarchy test, against 67.04 for the Gemma original, but Qwen3.5 4B beats it on Tau2-Bench (75.06 to 55.10) and on 64K-token recall. On an RTX 5070 it measured 548 ms to first token and 133.4 tokens a second; on a Jetson Orin Nano the wait is 5.8 seconds, too slow for live speech. Every E4B score is NVIDIA's own, run in its NeMo Gym harness.
Where it earns its place
A developer with a 32 GB NVIDIA workstation GPU
Run a browsing and terminal agent locally with screenshots in context
Use NVIDIA's 28 GB recipe: llama.cpp built from source, a 65,536-token context, q8_0 KV cache and the 1 GB DFlash drafter. NVIDIA reports 66.75 on BrowseComp and 50.98 on Terminal-Bench 2.1, but expect longer runs than Qwen3.6-35B-A3B.
A robotics team building a talking assistant on Jetson
Give a device spoken, tool-using replies without a cloud round trip
Run E4B's Q4_K_M build with conversational mode on and a thinking budget. NVIDIA measured 4.2 to 5.0 GB of memory and 13 to 15.5 tokens a second on a Jetson Orin Nano, with 5.8 seconds to first token, so a Jetson AGX Orin or RTX card suits live speech better.
A product lead at a PC software company
Ship an on-device assistant under a licence legal will sign off
OpenMDW-1.1 grants rights without restriction, with no revenue cap or use policy; only redistribution needs the licence and notices included. E4B adds Gemma 4's Apache 2.0 terms. The catch is hardware: both target NVIDIA GPUs only.
Before you commit
- Setup
- Build the latest llama.cpp from source with CUDA, download the 22.02 GB GGUF, 1.73 GB vision file and 1 GB drafter, and start llama-server with NVIDIA's 28 GB or 32 GB recipe. E4B runs from a prebuilt llama.cpp CUDA container with a 6.18 GB file.
- Integrations
- OpenAI-compatible chat completions with tool calls, JSON output and an enable_thinking switch. E4B adds conversational mode, a runtime thinking budget and asynchronous tool calling; DFlash drafters exist for both.
- Data and privacy
- Self-hosted, so data stays local. OpenMDW-1.1 allows commercial use with no revenue cap; rights end if you sue over patent or copyright in the model. NVIDIA publishes its pre- and post-training datasets but lists many sources as private.
- Cost at scale
- Weights are free and Artificial Analysis lists no metered price for the 74B. Hardware: a 32 GB GPU for 65K context or 48 GB for 262K on the 74B; E4B peaks at 3.1 to 5.0 GB on NVIDIA's tests, from an RTX 5070 to Jetson Orin.
- Limits
- 74B: Ada Lovelace and Blackwell only, Windows and Linux, video up to two minutes, high token use. E4B: text only, 64K context, and 5.8 seconds to first token on a Jetson Orin Nano.
What's good
- 21.4 on Artificial Analysis's index, fourth under 100B
- Image and two-minute video input in a 22 GB file
- OpenMDW-1.1: commercial use, no revenue cap
- E4B: 548 ms first token on an RTX 5070 (NVIDIA)
What's not
- 74B needs a 32 GB GPU, despite the name
- Slower than Qwen3.6-35B-A3B and Gemma 4 on NVIDIA's own tests
- 82K output tokens per AA index task
- E4B trails Qwen3.5 4B on Tau2-Bench and long context
Best for
Engineers with a 32 GB or larger NVIDIA GPU, or an RTX or DGX Spark, who want a capable local multimodal agent model, and device makers building voice assistants on Jetson or RTX.
Nemotron 3.5 Bolt (74B-A9B 24GB and E4B 4GB) alternatives
Nemotron 3.5 Bolt (74B-A9B 24GB and E4B 4GB): common questions
- Is Nemotron 3.5 Bolt (74B-A9B 24GB and E4B 4GB) worth it?
- The 74B puts a top-five sub-100B score on one workstation GPU, but the 24GB in its name is a weight budget, not the card you need. We rate it 3.9 out of 5.
- How much does Nemotron 3.5 Bolt (74B-A9B 24GB and E4B 4GB) cost?
- Free open weights under OpenMDW-1.1. No metered API price published for either model; E4B has a build.nvidia.com endpoint.
- What are the downsides of Nemotron 3.5 Bolt (74B-A9B 24GB and E4B 4GB)?
- 74B needs a 32 GB GPU, despite the name; slower than Qwen3.6-35B-A3B and Gemma 4 on NVIDIA's own tests; 82K output tokens per AA index task; e4B trails Qwen3.5 4B on Tau2-Bench and long context.
- Who is Nemotron 3.5 Bolt (74B-A9B 24GB and E4B 4GB) best for?
- Engineers with a 32 GB or larger NVIDIA GPU, or an RTX or DGX Spark, who want a capable local multimodal agent model, and device makers building voice assistants on Jetson or RTX.
- How much does Nemotron 3.5 Bolt (74B-A9B 24GB and E4B 4GB) cost for a team or in production?
- Weights are free and Artificial Analysis lists no metered price for the 74B. Hardware: a 32 GB GPU for 65K context or 48 GB for 262K on the 74B; E4B peaks at 3.1 to 5.0 GB on NVIDIA's tests, from an RTX 5070 to Jetson Orin.
- Is Nemotron 3.5 Bolt (74B-A9B 24GB and E4B 4GB) safe for company data?
- Self-hosted, so data stays local. OpenMDW-1.1 allows commercial use with no revenue cap; rights end if you sue over patent or copyright in the model. NVIDIA publishes its pre- and post-training datasets but lists many sources as private.
- What are the best Nemotron 3.5 Bolt (74B-A9B 24GB and E4B 4GB) alternatives?
- The closest alternatives we have reviewed are Nemotron 3.5 Super VL, Gemma 4, Underdog Saluki 27B 1.0 and Qwen3.8-27B.
Answers checked Oct 28, 2026.