What it is
Groq runs AI models through an API called GroqCloud. It built its name on its own chips, LPUs (Language Processing Units), and its model docs list gpt-oss-20b at about 1,000 tokens per second and gpt-oss-120b at about 500.
The company has changed shape. In December 2025 NVIDIA paid about $20 billion for a non-exclusive licence to Groq's chip technology and hired founder Jonathan Ross and much of the engineering team. Under new CEO Doug Wightman, Groq became an NVIDIA Cloud Partner in August 2026 and raised a $350 million Series A at a $3.5 billion valuation, repositioning as an inference cloud that runs NVIDIA hardware alongside its own.
The public model list is short: OpenAI's gpt-oss models, a Qwen model, Whisper speech-to-text and a few safety and voice models. Llama 3.1 8B and Llama 3.3 70B are still listed as production models but are Enterprise-only.
Our review
Speed is the reason to choose Groq, and it still delivers. The model docs list gpt-oss-20b at around 1,000 tokens per second and gpt-oss-120b at about 500, at $0.15 input and $0.60 output per million tokens for the larger model. For voice agents, live autocomplete or agents that chain a dozen calls per turn, that changes how a product feels, and the price is no premium over GPU clouds.
The catalogue is the cost of that speed. Production models are two gpt-oss sizes, two Whisper variants and two Llama models that now say "Contact Sales". Preview models, including qwen3.8-27b at $0.80 and $4.00, are marked for evaluation and "may be discontinued at short notice". A team that wants a specific frontier open-weight model, or fine-tuning, will find more on Together AI or Fireworks AI.
On data, the defaults are better than many rivals'. Groq says it does not retain inference inputs and outputs by default, though it may log them for up to 30 days for reliability and abuse monitoring; Zero Data Retention switches that off for any customer, not just enterprise accounts, at the cost of features that depend on stored data. The trust centre lists SOC 2 Type II reports for 2024 and 2025 and a HIPAA business associate addendum. Data that is retained sits in US Google Cloud buckets.
The company question has not gone away, it has changed. After NVIDIA licensed the LPU technology and hired its founder, Groq raised $350 million at $3.5 billion, roughly half its September 2025 valuation, and now describes itself as a neocloud that also sells NVIDIA capacity. That makes GroqCloud more likely to keep running and less likely to stay a pure LPU speed play. Cerebras is the closest rival on raw tokens per second, with a different hardware bet behind it.
Where it earns its place
A start-up building a phone-based voice agent for dental clinics
Keep the gap between caller and reply under a second
Whisper large-v3-turbo transcribes at $0.04 an audio hour and gpt-oss-20b answers at about 1,000 tokens a second. 10,000 call hours a month costs about $400 in transcription before LLM tokens.
A backend engineer whose agent makes 15 sequential LLM calls per user request
Cut end-to-end latency without rewriting the agent
Point the OpenAI-compatible client at GroqCloud and swap to openai/gpt-oss-120b. Speed per call is the win; the 131,072-token context and 65,536-token output cap are the constraints to design around.
A compliance lead at a telehealth company
Approve an inference provider for transcripts containing patient data
Groq offers a BAA, lists SOC 2 Type II reports, and lets admins turn on ZDR globally. Batch files are kept up to 30 days unless deleted, so batch pipelines need a deletion step.
Before you commit
- Setup
- Create a key in the GroqCloud console and use the OpenAI-compatible endpoint; the free plan needs no card.
- Integrations
- OpenAI-compatible chat and audio APIs; Batch and Flex processing on the paid Developer plan.
- Data and privacy
- No retention by default, up to 30 days of logging for abuse and reliability unless ZDR is on; batch files up to 30 days. SOC 2 Type II and HIPAA BAA per the trust centre; retained data stored in the US.
- Cost at scale
- A workload of 3 billion input and 1 billion output tokens a month on gpt-oss-120b costs $450 plus $600, about $1,050; on gpt-oss-20b it halves. Llama models are priced by sales.
- Limits
- Free plan: 30 requests a minute, 1,000 a day, 8,000 tokens a minute and 200,000 a day on gpt-oss and Qwen. Developer limits are shown only in your console.
What's good
- About 500 to 1,000 tokens per second on gpt-oss models
- Free tier with 1,000 requests a day on main models
- gpt-oss-120b at $0.15 in and $0.60 out per million tokens
- No retention by default; ZDR available to every customer
- Whisper large-v3-turbo at $0.04 per audio hour
What's not
- Six production models, two of them Enterprise-only Llama
- Preview models can be withdrawn at short notice
- Free tier capped at 8,000 tokens a minute on main models
- Business now pivoting from its own chips to NVIDIA capacity
Best for
Developers building voice, real-time or agent features where response speed matters more than model choice.
Alternatives
Groq: common questions
- Is Groq worth it?
- Groq still sells some of the fastest tokens on the market at bargain prices, but the company behind it is no longer the chip maker people signed up with, and its public catalogue has shrunk to fit. We rate it 4.0 out of 5.
- How much does Groq cost?
- Free plan with rate limits (30 requests/minute and 1,000/day for gpt-oss-120b). Pay-as-you-go Developer plan: gpt-oss-120b $0.15 in / $0.60 out per 1M tokens, gpt-oss-20b $0.075 / $0.30, qwen3.8-27b $0.80 / $4.00; Whisper large-v3-turbo $0.04/hour of audio. Llama and some newer models via sales.
- What are the downsides of Groq?
- Six production models, two of them Enterprise-only Llama; preview models can be withdrawn at short notice; free tier capped at 8,000 tokens a minute on main models; business now pivoting from its own chips to NVIDIA capacity.
- Who is Groq best for?
- Developers building voice, real-time or agent features where response speed matters more than model choice.
- How much does Groq cost for a team or in production?
- A workload of 3 billion input and 1 billion output tokens a month on gpt-oss-120b costs $450 plus $600, about $1,050; on gpt-oss-20b it halves. Llama models are priced by sales.
- Is Groq safe for company data?
- No retention by default, up to 30 days of logging for abuse and reliability unless ZDR is on; batch files up to 30 days. SOC 2 Type II and HIPAA BAA per the trust centre; retained data stored in the US.
- What are the best Groq alternatives?
- The closest alternatives we have reviewed are Cerebras, Fireworks AI, Together AI and OpenRouter.
Answers checked Oct 22, 2026.
Make Groq? Show this rating on your site
