OpenAI-compatible API
An OpenAI-compatible API is an endpoint from another provider or tool that accepts the same request and response format as OpenAI's API, so code written for OpenAI can switch to it by changing the base URL, key and model name.
Also called: OpenAI compatible endpoint
By Divya Sharma · Checked Nov 16, 2026 · How we review
OpenAI's Chat Completions format became the common way to talk to language models. Many other services copied it so developers would not have to rewrite their code. Open-source servers such as vLLM, Ollama and llama.cpp offer it, as do many inference providers and LLM gateways such as OpenRouter and LiteLLM.
How it works in practice
You keep using an OpenAI client library and point it elsewhere. For a local Ollama server, the documentation sets the base URL to `http://localhost:11434/v1/`; for vLLM, the server runs on a port you choose. You then pass that service's model name. The request body (messages, temperature, max tokens) and the response shape stay the same, which makes it easy to compare models or move between providers.
Compatible is not identical
Compatibility usually covers the core text endpoints and not everything else. Ollama's documentation says it supports a subset of the OpenAI API. Common gaps include structured output options, function calling details, image or audio inputs, newer endpoints, error codes and usage fields such as cached tokens. Parameters may be accepted but ignored. Test the features you rely on against each backend rather than assuming they behave like OpenAI.
Security
A self-hosted compatible server is a real network service. vLLM's documentation warns that its API key option does not protect every endpoint, so put such servers behind proper authentication and do not expose them directly to the internet.
Related terms
- LLM gateway
- An LLM gateway is a proxy server between your applications and model providers that exposes one API and handles keys, routing, fallbacks, rate limits, logging and spend tracking in a single place.
- Model serving
- Model serving is running a trained model as a service that applications call over a network, with software that loads the weights onto GPUs, queues and batches requests, manages memory and returns results.
- Inference provider
- An inference provider is a company that runs AI models on its own hardware and sells access to them through an API, usually charged per token. You send prompts and get outputs without managing GPUs yourself.
- API key
- An API key is a secret string an application sends with each request so that a service can tell which account is calling, apply its limits and bill it. Anyone holding the key can usually use it.
- Tool calling
- Tool calling, also called function calling, is a model API feature where you describe functions with a schema, the model replies with a structured request to call one, and your code runs it and returns the result.