What is Bifrost?
Bifrost is an open-source LLM gateway written in Go by Maxim AI, the company behind the Maxim evaluation and observability platform. It runs as a single HTTP service with a web UI, or as a Go library, and gives applications one OpenAI-compatible API in front of OpenAI, Anthropic, Bedrock, Vertex AI, Azure, Groq, Mistral, Ollama, vLLM and others; the docs' support matrix lists 32 providers.
The open-source build handles fallbacks, load balancing across keys, semantic caching, an MCP gateway for tool servers, Prometheus metrics and virtual keys with budgets and rate limits per team or customer. It also offers drop-in endpoints for the OpenAI, Anthropic and Google GenAI SDKs, so existing code changes only its base URL.
The whole repository is Apache 2.0. A separately licensed Enterprise build adds clustering, adaptive load balancing, SSO with role-based access, audit logs, vault integration, guardrails and VPC deployment, with a 14-day trial and no published price.
Is Bifrost worth it?
Bifrost is the most serious challenger to LiteLLM among self-hosted gateways, and its case rests on speed. Its README reports 11 µs of added latency per request at a sustained 5,000 requests a second on an AWS t3.xlarge, and 59 µs on a t3.medium. Maxim's comparison page goes further: at 500 requests a second on a t3.medium it shows Bifrost completing 100% of requests against LiteLLM's 88.78%, with a P99 latency of 1.68 seconds against 90.72. These are the vendor's own tests, and the comparison charts are simulated from measured distributions. The project is young, created in March 2025, but busy: about 9,100 stars and a gateway release, v2.3.2, on 20 October 2026.
The licence is cleaner than LiteLLM's. The entire repository is Apache 2.0, with no commercial folder, and the docs put virtual keys, dollar budgets with daily to yearly resets, token and request limits, and team and customer hierarchies in the open-source build. The line blurs elsewhere: the enterprise page lists governance and guardrails as paid features while the open-source description claims both, and the README says enterprise deployments add the MCP gateway that the docs also present as standard. Clustering and adaptive load balancing are clearly paid, which matters to anyone planning high availability.
The weak point is security defaults. Management authentication was off by default, and the official Docker image binds to all interfaces, so anyone who could reach the port could administer the gateway. In September 2026 JFrog disclosed CVE-2026-90898 (GitHub advisory GHSA-86gf-xh3g-rvxq), rated 9.8 out of 10 in both the NVD record and the advisory: one unauthenticated request could register a stdio MCP server and run commands inside the container. It is fixed in transports v2.1.0, released on 8 September; The Hacker News reports that Maxim considers the flaw low-severity, while JFrog advises treating any exposed instance as compromised and rotating every key. Two high-severity advisories sit beside it, a plugin-loading code execution flaw on dynamically linked builds and an incomplete SSRF filter in July. The setup guide now requires a setup token for the first admin account and warns that a new instance has no password.
Coverage is the other gap. The docs list 32 providers against LiteLLM's 100+, the README still says 23+, and the repository description claims 10,000+ models, which counts models rather than integrations. There is no Python library, only a Go SDK. Almost every Bifrost-versus-LiteLLM article online is published by Maxim or its staff; the one independent view we found, a Hacker News comment from late 2025, called it "fast but many features paywalled". The speed lead is real in the vendor's numbers but matters mainly at thousands of requests a second: in the same README table, average request time including the provider was 1.61 seconds, against 11 µs of gateway overhead.
Where it earns its place
A platform engineer whose Python gateway needs a fleet of nodes at peak
Serve several thousand requests a second through fewer gateway instances
Maxim reports 5,000 requests a second with 100% success on a 2 vCPU, 4 GB t3.medium, peaking at about 1.3 GB of memory. Run your own load test with the published benchmark guide before resizing, and back multiple instances with PostgreSQL 16 or later.
A backend team writing services in Go
Call several providers with fallbacks without running a separate proxy
go get github.com/maximhq/bifrost/core embeds the router in the service, so there is no extra network hop and no gateway to secure. The trade is that you lose the web UI and shared spend tracking a central gateway gives you.
An engineering manager at a 60-person company with three product teams
Give each team its own key and monthly budget without buying an enterprise licence
Create a virtual key per team with a dollar budget that resets monthly and request limits, all in the open-source build. SSO for the dashboard, audit logs and clustering need Enterprise, which has a 14-day trial and unpublished pricing.
A team building agents that call internal tools
Register MCP servers once and control which keys can use which tools
Add MCP clients in the gateway and filter tools per virtual key. Turn on management authentication first and run v2.1.0 or later: stdio MCP registration was the route for the 9.8-rated flaw.
Before you commit
- Setup
- npx -y @maximhq/bifrost or docker run -p 8080:8080 maximhq/bifrost starts the gateway and web UI on port 8080, which the README says takes under a minute; config and logs default to SQLite in the app directory, with PostgreSQL 16+ for production, plus Helm charts and Terraform in the repository.
- Integrations
- OpenAI-compatible API plus drop-in OpenAI, Anthropic, Bedrock and GenAI SDK endpoints; 32 providers in the docs; MCP gateway; Prometheus and OpenTelemetry; a plugin for Maxim's own observability product; a Go SDK.
- Data and privacy
- Self-hosted, so prompts go only to the providers you configure; we found no usage reporting to Maxim in the docs, and its telemetry plugin serves Prometheus metrics locally. Content logging can be switched off per key. One critical and two high-severity advisories were published in 2026; management auth must be enabled by hand.
- Cost at scale
- The vendor's 5,000 requests-a-second test ran on a t3.medium (2 vCPU, 4 GB) and a t3.xlarge (4 vCPU, 16 GB), the latter peaking at 3.3 GB of memory. Running more than one node with shared state means PostgreSQL; peer-to-peer clustering is Enterprise, priced on request.
- Limits
- 550 open issues and 823 open pull requests; frequent releases with database migrations; PostgreSQL migrations can fail behind PgBouncer in transaction mode or RDS Proxy; no Python SDK; fewer providers than LiteLLM.
What's good
- Apache 2.0 across the whole repository
- Virtual keys, budgets and rate limits in the free build
- 11 µs overhead at 5,000 requests a second (vendor benchmark)
- Drop-in endpoints for OpenAI, Anthropic and GenAI SDKs
What's not
- Admin API unauthenticated by default before v2.1.0; CVSS 9.8 flaw
- 32 providers against LiteLLM's 100+
- Clustering and adaptive load balancing are Enterprise-only
- Most comparisons online are written by the vendor
Best for
Teams running high request volumes, or Go shops, that want an Apache-licensed gateway with budgets and are prepared to lock down its admin API.
Bifrost alternatives
Head to head
Bifrost: common questions
- Is Bifrost worth it?
- A fast, Apache-licensed gateway with budgets in the free build, let down by a security default that left its admin API open until September 2026. We rate it 3.9 out of 5.
- How much does Bifrost cost?
- Free and open source under Apache 2.0. Enterprise is custom-priced, with a 14-day free trial; the price is not published.
- What are the downsides of Bifrost?
- Admin API unauthenticated by default before v2.1.0; CVSS 9.8 flaw; 32 providers against LiteLLM's 100+; clustering and adaptive load balancing are Enterprise-only; most comparisons online are written by the vendor.
- Who is Bifrost best for?
- Teams running high request volumes, or Go shops, that want an Apache-licensed gateway with budgets and are prepared to lock down its admin API.
- How much does Bifrost cost for a team or in production?
- The vendor's 5,000 requests-a-second test ran on a t3.medium (2 vCPU, 4 GB) and a t3.xlarge (4 vCPU, 16 GB), the latter peaking at 3.3 GB of memory. Running more than one node with shared state means PostgreSQL; peer-to-peer clustering is Enterprise, priced on request.
- Is Bifrost safe for company data?
- Self-hosted, so prompts go only to the providers you configure; we found no usage reporting to Maxim in the docs, and its telemetry plugin serves Prometheus metrics locally. Content logging can be switched off per key. One critical and two high-severity advisories were published in 2026; management auth must be enabled by hand.
- What are the best Bifrost alternatives?
- The closest alternatives we have reviewed are LiteLLM, Portkey AI Gateway and vLLM Semantic Router.
Answers checked Oct 26, 2026.