What it is
Baseten hosts AI models for companies that need them to run reliably in production. It offers Model APIs, a shared pay-per-token API for open models such as GPT OSS 120B, DeepSeek V4.1, GLM-5.3 and Kimi K3, and dedicated deployments where your own or a fine-tuned model runs on GPUs reserved for you. It also runs training jobs on the same hardware.
The Basic plan has no monthly fee; you pay for usage, and new accounts get credits of an unstated amount. On Model APIs, GPT OSS 120B costs $0.10 input and $0.50 output per million tokens. Dedicated GPUs are billed by the minute, from about $0.63 an hour for a T4 to about $6.50 for an H100 and about $9.98 for a B200. Enterprise customers can run Baseten in their own cloud account.
Our review
Baseten is aimed at teams past the prototype stage whose models now carry real traffic. Its security practices commit both Model APIs and dedicated deployments to zero data retention: synchronous inputs and outputs are not logged or written to persistent storage, and customer content is not used for training. Async requests are the exception, with inputs held only until processing completes. SOC 2 Type II and HIPAA apply even on the free Basic plan, which is unusual and matters to health and finance buyers.
The shared Model APIs are cheap enough to be a default. GPT OSS 120B at $0.10 in and $0.50 out per million tokens and DeepSeek V4.1 Flash at $0.30 and $1.20 are at the low end of hosted open-model pricing, and cached input is billed at a tenth of the normal rate on most models. Since August 2026 Baseten is also an Inference Provider on Hugging Face.
Dedicated hardware is where the bill grows. An H100 works out to about $6.50 an hour on per-minute billing, against about $3.95 on Modal, and an A100 costs $4.00 against Modal's $2.50. Running one H100 around the clock comes to roughly $4,680 a month on Baseten. Priority access to scarce GPUs and higher Model API rate limits require the quote-based Pro plan, and the Basic plan's rate limits are not published.
What the premium buys is engineering. Models are packaged with Truss, Baseten's open-source config format, pushed with one CLI command, and promoted to a stable endpoint URL between versions. Enterprise adds deployment into your own VPC or a hybrid of both clouds, and lets you spend existing cloud commitments, which is often what moves the decision for a company that has already prepaid AWS or Google.
Where it earns its place
An ML engineer at a 40-person health-tech startup
Serve a fine-tuned clinical note model under HIPAA
Package the model with Truss, push to a dedicated H100 and promote to a stable endpoint. HIPAA and SOC 2 Type II apply on Basic; one H100 running continuously is about $4,680 a month, less if traffic lets replicas scale down.
A backend team adding an LLM feature to a B2B app
Call a capable open model without running GPUs
Model APIs with GPT OSS 120B: 2 billion input and 500 million output tokens a month cost about $450, with zero data retention on synchronous calls.
A head of infrastructure at a bank with an AWS commitment
Host open models without data leaving the bank’s cloud account
Enterprise deploys Baseten into the bank’s own VPC and draws down existing cloud commitments; pricing is by quote and not published.
Before you commit
- Setup
- Install the Truss CLI, describe the model in config.yaml and run baseten model push; a development deployment with live reload comes up first, then a production deployment you promote.
- Integrations
- Truss also wraps existing vLLM, SGLang or Triton containers; Model APIs are available through Hugging Face Inference Providers.
- Data and privacy
- Zero data retention for synchronous Model API and dedicated inference calls, no training on customer content; SOC 2 Type II and HIPAA on all plans; data residency control and self-hosting on Enterprise.
- Cost at scale
- One H100 at $0.10833 a minute is about $4,680 a month if never idle, against about $2,844 on Modal; per-minute billing with no charge for idle time narrows that gap for spiky traffic.
- Limits
- Basic runs on Baseten Cloud only, without priority GPU access; Model API rate limits are higher on Pro but not published for either plan.
What's good
- Zero data retention on Model APIs and dedicated inference
- SOC 2 Type II and HIPAA on every plan, including free Basic
- GPT OSS 120B at $0.10 / $0.50 per million tokens
- Enterprise runs in your VPC and uses existing cloud commitments
What's not
- Dedicated H100s cost about 65% more per hour than Modal
- Free credit amount and Basic rate limits not published
- Priority GPU access needs the quote-based Pro plan
- Async requests temporarily store inputs
Best for
Companies serving custom or fine-tuned models in production that need zero data retention, HIPAA and the option to run in their own cloud.
Alternatives
Baseten: common questions
- Is Baseten worth it?
- Baseten sells production inference with the compliance paperwork already done, and charges a clear premium for GPUs you could rent cheaper elsewhere. We rate it 4.2 out of 5.
- How much does Baseten cost?
- Basic $0/month, pay as you go; Pro and Enterprise by quote. Model APIs per 1M tokens, e.g. GPT OSS 120B $0.10 in / $0.50 out, Kimi K3 $3 / $15. Dedicated GPUs per minute: T4 about $0.63/hour, A100 about $4.00/hour, H100 about $6.50/hour, B200 about $9.98/hour.
- What are the downsides of Baseten?
- Dedicated H100s cost about 65% more per hour than Modal; free credit amount and Basic rate limits not published; priority GPU access needs the quote-based Pro plan; async requests temporarily store inputs.
- Who is Baseten best for?
- Companies serving custom or fine-tuned models in production that need zero data retention, HIPAA and the option to run in their own cloud.
- How much does Baseten cost for a team or in production?
- One H100 at $0.10833 a minute is about $4,680 a month if never idle, against about $2,844 on Modal; per-minute billing with no charge for idle time narrows that gap for spiky traffic.
- Is Baseten safe for company data?
- Zero data retention for synchronous Model API and dedicated inference calls, no training on customer content; SOC 2 Type II and HIPAA on all plans; data residency control and self-hosting on Enterprise.
- What are the best Baseten alternatives?
- The closest alternatives we have reviewed are Modal, Together AI, Fireworks AI and Replicate.
Answers checked Oct 22, 2026.
Make Baseten? Show this rating on your site
