Cerebras vs Groq vs SambaNova: where to run open models fast, and what it costs
By Sheela ·
If your product makes a person wait for a model, or chains a dozen model calls before it answers, three companies sell open-model tokens faster than the GPU clouds: Cerebras, Groq and SambaNova. All three built their own chips to do it. They differ in what they serve, what they charge, how much you can buy without a contract, and how steady the company behind each one looks. This guide is for a team deciding where to run open models in production.
Every figure below comes from our reviews of Cerebras, Groq and SambaNova Cloud, from the vendors' own pages, or from Artificial Analysis, which times every provider serving the same model; we read all of them on 7 November 2026. We have not run these services ourselves.
Three chips, one problem
A language model writes one token at a time, and for every token the chip has to read the model's weights from memory. On a GPU those weights sit in high-bandwidth memory (HBM) beside the processor, and moving them, rather than the arithmetic, is usually what limits how fast one user's answer arrives. Each of the three attacks that step in a different way.
- Cerebras: one chip the size of a wafer. Chipmakers normally cut a silicon wafer into dozens of separate chips. Cerebras keeps the whole wafer as one processor, so far more memory and compute sit on the same piece of silicon and data does not have to cross between chips. Its WSE-3 has 4 trillion transistors and 900,000 cores on 46,225 mm², a square about 21.5 cm on each side.
- Groq: memory on the chip, timing planned in advance. Groq's Language Processing Unit keeps weights in on-chip SRAM instead of HBM: 230 MB per chip, with bandwidth Groq puts above 80 TB/s against about 8 TB/s for GPU memory. Its compiler schedules every step down to the clock cycle, so nothing waits on a cache. The trade-off is that one chip holds only a sliver of a large model, so each model is spread across many linked chips.
- SambaNova: a reconfigurable assembly line. SambaNova's Reconfigurable Dataflow Unit lets the compiler lay a whole stretch of the model across the chip, so results flow from one step to the next without a round trip to memory. Its SN40L pairs 520 MiB of on-chip memory with 64 GiB of HBM and up to 1.5 TiB of ordinary DDR memory, which lets one system hold many models and switch between them quickly.
What you buy is speed for each user, not cheaper tokens. Each chip also needs its own compiler work for every model, and in our reading that is one reason all three menus are short.
The same models, side by side
Only one model is on all three shared APIs: OpenAI's gpt-oss-120b. That makes it the fairest comparison, with the two GPU clouds we have reviewed for scale. Output speed is Artificial Analysis's headline figure, tokens per second over the whole request, averaged across short, medium and long prompts over the previous five days. Because gpt-oss-120b reasons before it answers, the last column, time to the first token of the actual answer, is closer to what a user feels.
| gpt-oss-120b on | Input $/1M | Output $/1M | Output speed (tokens/s) | First token (s) | First answer token (s) |
|---|---|---|---|---|---|
| Cerebras | $0.35 | $0.75 | 1,347 | 0.65 | 2.26 |
| Groq | $0.15 | $0.60 | 338 | 1.02 | 4.90 |
| SambaNova | $0.22 | $0.59 | 417 | 2.56 | 6.74 |
| Together AI | $0.15 | $0.60 | 65 | 1.53 | 46.23 |
| Fireworks AI | $0.15 | $0.60 | Not measured | Not measured | Not measured |
Prices are each vendor's list price (Cerebras, Groq, SambaNova, Fireworks, and our Together AI review); speeds are from Artificial Analysis, read on 7 November 2026. Artificial Analysis does not list Fireworks for this model.
Three things stand out. Cerebras is three to four times as fast as Groq or SambaNova and starts sooner, but charges more than twice Groq's input price. SambaNova writes faster than Groq once it gets going, yet takes more than twice as long as Groq to produce its first token, which matters for anything spoken aloud. And the vendors' own figures are higher than the measured ones: Cerebras's docs say about 3,000 tokens a second and Groq's about 500, partly because Artificial Analysis counts the whole request, including the wait before the first token.
Artificial Analysis also checks whether each host's answers match OpenAI's reference version of the model. SambaNova scores 98.5%, Cerebras 91.8%, Groq 90.3% and Together AI 81.1%. The ranges around those scores are wide (Groq's runs from 82.7% to 97.9%), so read them as a reason to test your own prompts, not as a ranking.
The other overlaps are partial:
| Model | Where | Input / output $ per 1M | Speed (tokens/s) | First token (s) |
|---|---|---|---|---|
| Qwen 3.8 27B | Cerebras | $0.99 / $1.49 | about 1,850 (vendor) | Not measured |
| Qwen 3.8 27B | Groq (preview) | $0.80 / $4.00 | about 450 (vendor) | Not measured |
| Llama 3.3 70B | Groq (Enterprise only) | Contact sales | 211 | 1.34 |
| Llama 3.3 70B | SambaNova | $0.60 / $1.20 | 121 | 1.13 |
| GLM-5.3 | SambaNova | $2.80 / $8.80 | Not measured | Not measured |
| GLM-5.3 | Together AI | $1.40 / $4.40 | 159 | 1.08 |
| GLM-5.3 | Fireworks AI (Fast tier) | $2.10 / $6.60 | 207 | 1.53 |
Measured figures are from Artificial Analysis's pages for Llama 3.3 70B and GLM-5.3; "vendor" means the provider's own docs. Fireworks' standard GLM-5.3 tier is $1.40 and $4.40. The GLM-5.3 rows hold a warning: SambaNova charges twice Together AI's price for it, and Artificial Analysis has no measurement of SambaNova's GLM-5.3 to show what the premium buys.
What a month costs
Take a production workload of 2 billion input and 500 million output tokens a month on gpt-oss-120b. At list prices it costs about $1,075 on Cerebras, $735 on SambaNova and $600 on Groq, Together AI or Fireworks AI. If the work can wait, Fireworks' batch tier is 50% off, which brings it to about $300, and Together AI's batch discount goes up to 50%.
The limits that bite in production
Price tables hide the ceilings that decide whether you can actually ship.
- Cerebras. The $5 trial credit lasts 30 days, allows 5 requests a minute and cuts context to 65k tokens; when it ends, access stops until you buy credits. Pay-as-you-go allows 1,000 requests and 1 million uncached tokens a minute on gpt-oss-120b with no daily cap, but Qwen 3.8 27B stays at 300 requests a minute. Context is 131k tokens on gpt-oss-120b. Anything beyond the two shared models means Dedicated Inference and an agreement.
- Groq. The free plan allows 30 requests a minute, 1,000 a day and 8,000 tokens a minute on the main models. Paid limits are shown only in your console, not published. gpt-oss-120b has a 131,072-token context and caps output at 65,536 tokens; the Qwen preview caps output at 16,384. Both Llama models are Enterprise only.
- SambaNova. The free tier allows 20 requests a day per model. With a card, the Developer tier allows 60 requests a minute and 12,000 a day on most models, and 20 million tokens a day in total. The workload above needs about 83 million tokens a day, four times that cap, so it needs an Enterprise subscription at a quoted price. Llama 3.3 70B caps each answer at 3,072 tokens.
All three label some models "preview" and say they can be withdrawn at short notice. SambaNova's documentation and its live model list disagreed on the day we checked, so test against the model list endpoint, not the docs.
What happens to your data
- Cerebras says it does not retain prompts, requests or outputs from the inference API, though prompt-cache entries live from 5 minutes to an hour. A SOC 2 Type 2 report and a data processing agreement (DPA) are available on request; the DPA covers business accounts.
- Groq does not keep inputs and outputs by default but may log them for up to 30 days for reliability and abuse monitoring, unless you switch on Zero Data Retention, which any customer can do. It lists SOC 2 Type II reports and offers a HIPAA business associate addendum. Batch files are kept up to 30 days unless deleted.
- SambaNova says on its product page that it never collects your prompts, but its licence agreement sets no retention period and lets it use query logs to improve the service, and we found no public SOC 2 report. For the strictest cases it sells the same hardware for your own data centre as SambaStack, with air-gapped installs since March 2026.
For health or other regulated data, Groq is the easiest of the three to approve from published documents. For data that must not leave your building, SambaNova sells the same hardware as its cloud as a standard on-premises product.
Where each one wins
Agents that chain many calls: Cerebras
At 1,347 tokens a second and 0.65 seconds to the first token, Cerebras shortens every step of a coding or research agent, and dozens of steps per task add up to minutes saved. 1,000 requests a minute on pay-as-you-go is rarely the bottleneck. You pay for it: more than twice Groq's input price, and only two models on the shared API.
Voice: Groq
A phone agent has to hear, think and answer before the caller notices a pause. Groq sells Whisper large-v3-turbo transcription at $0.04 an audio hour next to gpt-oss-20b, which Artificial Analysis measures at 630 tokens a second for $0.075 in and $0.30 out, so one vendor and one contract cover both halves. Add the BAA and Zero Data Retention, and Groq is the practical choice for contact centres.
A cheaper second source of fast gpt-oss: SambaNova
SambaNova sits between the other two on price and speed for gpt-oss-120b, scores best on answer accuracy, and serves models neither rival offers, such as GLM-5.3 and MiniMax-M3.1-Flash with a 1-million-token context. Its small self-serve limits make it a fallback rather than a primary, unless you are buying the hardware.
Batch cost and model choice: a GPU cloud
None of the three is the cheap option for work nobody is waiting on. Together AI and Fireworks AI match Groq's gpt-oss-120b price, cut it by up to half in batch, carry far longer model lists, fine-tune models for you, and rent dedicated GPUs (an H100 is $5.49 an hour on Together AI and $8 on Fireworks). If you need a particular open model, a fine-tune, or steady throughput across many users rather than speed for one, a GPU cloud is the better buy. Self-hosting on vLLM or SGLang wins at steady high volume, as our inference serving guide explains.
Lock-in and supply risk
Switching is cheap in code. All three speak the OpenAI chat format, so moving is mostly a base URL and a model name, and OpenRouter lists all three as providers of gpt-oss-120b. The gaps are in the details: Groq rejects logprobs and logit_bias, and Cerebras limits n to 1 and will not combine tools with a response schema on gpt-oss-120b. Test the exact parameters your code sends.
The bigger risk is the companies, and 2026 has moved all three.
- Cerebras listed on Nasdaq in May 2026 and signed an OpenAI compute deal worth over $10 billion in January. In late September, research firm SemiAnalysis reported that the Ultrafast mode of OpenAI's GPT-6.1 Sol runs on Nvidia GPUs, not Cerebras; the shares fell nearly 20% in a week, to a value just over $39 billion against $95 billion at the end of the first trading day, CNBC reported. Oracle plans to offer Cerebras inference on its cloud in 2027. Its fixed-price Cerebras Code plans are listed as sold out.
- Groq licensed its chip technology to Nvidia for about $20 billion in December 2025, and Nvidia hired its founder and much of the engineering team. Groq now runs Nvidia hardware alongside its own as an Nvidia Cloud Partner, raised $350 million at $3.5 billion in August, and is seeking a new round at $8 billion, Bloomberg reported on 6 November. GroqCloud looks likely to keep running; whether it stays an LPU service is less certain.
- SambaNova dropped AI training in April 2025 and laid off 77 people, about 15% of staff. Intel discussed buying it for about $1.6 billion in late 2025; instead Intel joined a $350 million round in February 2026, and in July SambaNova raised $1 billion at an $11 billion valuation, with its SN50 chip due to ship in the second half of 2026.
Open weights are your insurance. A team on gpt-oss-120b can move between the five providers in the first table by changing a URL and a model name; a team that builds on a model only one fast vendor serves cannot.
A starting setup
- Default traffic on a GPU cloud. gpt-oss-120b or the open model you need on Together AI or Fireworks AI, with batch for anything offline.
- Latency-critical calls on a fast chip. Agent steps to Cerebras, voice to Groq, through the same OpenAI client or through OpenRouter.
- A configured fallback. SambaNova for gpt-oss-120b, so an outage or a rate limit at one vendor does not take your product down.
- A monthly check. Prices, preview labels and model lists change often at all three; re-read them, and re-run your own quality checks after any switch.
The catch: the speed numbers here come from one benchmark service sending mostly single requests from one place, and they say little about how a provider behaves at your peak load. The vendors' own figures are higher, the measured ones move week to week, and only one model is on all three menus. Treat the tables as a shortlist, then run your own prompts at your own volume before you sign anything.
Our take: the fast-chip vendors are the express checkout. They are worth it when a person is waiting with a few items and a train to catch, and wasteful for the weekly shop. Send the calls someone is waiting on to Cerebras or Groq, keep SambaNova as the second till and the option on hardware you own, and let a GPU cloud carry everything else.