What it is
LiteLLM does two jobs. As a Python library it lets code call OpenAI, Anthropic, Gemini, Bedrock, Azure, Vertex AI and 100+ other providers with one OpenAI-format function. As a proxy server, which the project now calls its AI Gateway, it sits between a whole organisation and those providers.
The gateway issues virtual keys, tracks spend by key, user and team, enforces budgets and rate limits, load-balances across deployments and falls back to another model when one fails. It logs to Langfuse, MLflow, Helicone and other tools, and can run guardrails such as PII masking. The repository description now advertises a Rust core alongside the Python SDK, and the docs link a Rust gateway in beta.
It is open core. Everything outside the enterprise/ folder is MIT; that folder has its own commercial licence. The enterprise tier adds SSO and SCIM, audit logs, role-based access control, air-gapped deployment and support SLAs, at prices given only on request.
Our review
LiteLLM is the gateway most teams reach for first, and its claim to that position is breadth: more than 100 providers behind one OpenAI-format API, a release every few days (v1.106.2 shipped on 21 October 2026) and about 61,900 GitHub stars. Many tools ship a LiteLLM integration, so it rarely becomes the odd part of a stack. The README's 8 ms P95 latency at 1,000 requests a second is the project's own benchmark.
Not every team needs a gateway. If you call one or two providers from one service, the providers' own SDKs and a retry wrapper are simpler and remove a hop. A gateway starts paying for itself when several teams share provider keys, when finance wants spend per team, or when you need automatic failover between, say, Azure OpenAI and OpenAI. Virtual keys, budgets and fallbacks all work in the open-source build.
Its weakness is that a gateway holds every provider key you own, which makes its security record matter. On 24 March 2026 attackers with a stolen PyPI token published two poisoned releases, 1.82.7 and 1.82.8, that harvested SSH keys, cloud credentials and Kubernetes tokens; LiteLLM's own incident note says they were live for about 40 minutes and traces the breach to the Trivy scanner in its CI pipeline, and the official Docker images were not affected. The project's own advisory list is long too: 17 published in 2026, four rated critical, including an authentication bypass via Host header injection in May and a privilege escalation to proxy admin in September. The mitigations are there, as -stable Docker images get 12-hour load tests and every image is cosign-signed, but they only help teams that pin and verify.
The open-core line is also blurry. SSO, audit logs and SLAs are clearly paid; the enterprise page lists virtual keys, budgets and RPM/TPM limits under its governance features too, while the open-source docs describe them as standard. The same page cites SOC 2 Type 2 and ISO 27001 and a one-hour response target for Sev 0 outages. Portkey's gateway is the main open-source rival, and OpenRouter suits teams that would rather rent the layer than run it, at the price of sending every prompt through a third party.
Where it earns its place
A platform team giving 300 engineers access to OpenAI, Anthropic and Bedrock
Issue per-team keys with budgets and report spend by team
Run the proxy on Kubernetes with Postgres and Redis, hand each team a virtual key with a budget, and give finance spend by key, user and team. Free in the open-source build; SSO for the admin UI needs an enterprise licence.
An SRE responsible for an LLM-backed product's uptime
Fail over from Azure OpenAI to OpenAI when one deployment degrades
Put both deployments in one model group with fallbacks and retries in the router. The cost is an extra network hop and a component that holds every provider key, so pin a -stable image and verify its cosign signature.
A head of engineering at a regulated fintech
Decide between self-hosted open-source LiteLLM and the enterprise licence
Enterprise is custom-priced with a 30-day self-hosted trial key and adds SSO, SCIM, audit logs and a one-hour Sev 0 response target. Either way, budget time for patching: four critical advisories landed in 2026.
Before you commit
- Setup
- uv tool install 'litellm[proxy]' then litellm --model gpt-4o runs a local proxy on port 4000; production uses the Docker image, the Helm chart or Terraform modules for AWS (ECS Fargate, Aurora, ElastiCache) and GCP (Cloud Run, Cloud SQL).
- Integrations
- OpenAI-compatible /chat/completions, /responses, /embeddings, /messages, /batches and more across 100+ providers; logging to Langfuse, MLflow, Helicone and others; MCP and A2A gateways; a Python SDK that works without the proxy.
- Data and privacy
- Self-hosted, so prompts go only to the providers you configure; the production guide mentions no telemetry to BerriAI. The proxy keeps keys, spend logs and optionally prompts in your Postgres. Seventeen security advisories were published in 2026, four critical.
- Cost at scale
- The docs size each worker at 1 vCPU and 4 GiB, so an 8-worker node needs 8 vCPU and 32 GiB, plus Postgres and Redis 7 once you run more than one instance. Enterprise is priced on request.
- Limits
- 5,636 open issues and pull requests; releases every few days mean frequent upgrades; Postgres connection pools must be sized per instance and worker; the Rust gateway is still in beta.
What's good
- 100+ providers in OpenAI format
- Virtual keys, budgets and fallbacks in the free build
- Cosign-signed -stable images with 12-hour load tests
- Terraform modules for AWS and GCP
What's not
- 17 security advisories in 2026, four critical
- Poisoned PyPI releases in March 2026
- Open-core boundary is unclear in the docs
- Enterprise pricing not published
Best for
Platform teams that must give many developers governed access to many model providers from their own infrastructure.
Alternatives
LiteLLM: common questions
- Is LiteLLM worth it?
- The default self-hosted gateway for teams that need one API, per-team budgets and fallbacks across many providers, provided they pin versions and treat it as security-critical infrastructure. We rate it 4.3 out of 5.
- How much does LiteLLM cost?
- Free and open source under MIT, except the enterprise/ folder, which is under a separate commercial licence. Enterprise is custom-priced with a free 30-day trial key.
- What are the downsides of LiteLLM?
- 17 security advisories in 2026, four critical; poisoned PyPI releases in March 2026; open-core boundary is unclear in the docs; enterprise pricing not published.
- Who is LiteLLM best for?
- Platform teams that must give many developers governed access to many model providers from their own infrastructure.
- How much does LiteLLM cost for a team or in production?
- The docs size each worker at 1 vCPU and 4 GiB, so an 8-worker node needs 8 vCPU and 32 GiB, plus Postgres and Redis 7 once you run more than one instance. Enterprise is priced on request.
- Is LiteLLM safe for company data?
- Self-hosted, so prompts go only to the providers you configure; the production guide mentions no telemetry to BerriAI. The proxy keeps keys, spend logs and optionally prompts in your Postgres. Seventeen security advisories were published in 2026, four critical.
- What are the best LiteLLM alternatives?
- The closest alternatives we have reviewed are Portkey AI Gateway, vLLM Semantic Router and Langfuse.
Answers checked Oct 23, 2026.
Make LiteLLM? Show this rating on your site
