What it is
SGLang is a serving framework for language, vision-language and diffusion models, hosted by the non-profit LMSYS. Its best-known feature is RadixAttention, which reuses work across requests that share the same beginning, such as a long system prompt.
It offers an OpenAI-compatible server, multi-GPU and multi-node parallelism, quantisation, speculative decoding and structured output. It runs on NVIDIA GPUs from RTX cards to B300 and GB300, AMD Instinct MI300X to MI355X, Google TPUs, Intel and Huawei Ascend hardware, and it is the rollout engine inside RL training frameworks such as verl, slime and AReaL.
Releases land roughly every two weeks; v0.5.22 was published on 19 October 2026.
Our review
SGLang is the strongest alternative to vLLM for prefix-heavy serving, and the one whose security posture needs the most care. It has about 37,700 GitHub stars and ships a release every two weeks or so. RadixAttention keeps a tree of cached prompt prefixes, so repeated system prompts, few-shot examples and multi-turn histories are not recomputed for every request.
That design suits agents and reinforcement learning. Agent loops resend the same long context many times, and RL training generates huge numbers of rollouts from shared prompts, which is why verl, slime, AReaL and Google's Tunix use SGLang for generation. Hardware coverage is now wide, from consumer RTX cards to B300s, MI355X, TPU v7 and Ascend NPUs, and NVIDIA Dynamo, llm-d and Ray Serve can orchestrate it.
Security is the weak point. In 2026 CERT/CC published notes on several remote code execution flaws, including CVE-2026-5760, a template injection through a malicious GGUF model file rated 9.8, and in May Antiproof disclosed three more (CVE-2026-7301, 7302 and 7304), two involving unsafe pickle deserialisation, saying the maintainers had not responded and no patch existed. The template bug has since been fixed in the code, but SGLang's own serving examples bind to 0.0.0.0, and the server should never face an untrusted network without authentication in front of it.
The trade-offs against vLLM are familiar. Tuning assumes serving experience and the tracker carries about 6,000 open issues and pull requests. How much RadixAttention saves depends on how long the shared prefix in your own traffic is, which makes a benchmark on real requests worth more than any published chart.
Where it earns its place
A platform engineer serving an open model to an internal agent fleet on one 8-GPU H100 node
Cut latency and GPU cost when every agent call repeats a long system prompt and tool list
Run the lmsysorg/sglang Docker image with the OpenAI-compatible server and point the agents at it; RadixAttention reuses the shared prefix across calls. The software is free; the cost is the node, which must sit on a private network behind authentication.
An ML researcher running RL post-training with verl or slime
Generate rollouts fast enough to keep training GPUs busy
Both frameworks use SGLang as the generation engine, sharing prompt prefixes and updating weights in place between steps. Free; the weight-update endpoints are where earlier deserialisation bugs sat, so keep them off shared networks.
An infrastructure lead at a company standardised on AMD MI300X
Self-host a chat model without depending on CUDA-only software
SGLang lists MI300X through MI355X as supported platforms with dedicated docs. No licence fees under Apache 2.0; enterprise support means contacting LMSYS, with no published price.
Before you commit
- Setup
- docker pull lmsysorg/sglang:latest, or uv pip install sglang on Python 3.11 or later, then launch the server with a Hugging Face model path; needs a supported GPU, TPU or NPU with memory for the weights and KV cache.
- Integrations
- OpenAI-compatible API; deployment through NVIDIA Dynamo, llm-d and Ray Serve; RL frameworks verl, slime, AReaL, Tunix and Miles; SpecForge for training speculative-decoding draft models.
- Data and privacy
- Runs entirely on your hardware with no hosted service. Several 2026 CVEs allow remote code execution when HTTP or ZMQ ports are reachable or untrusted model files are loaded, so bind to a private network, add authentication and load models only from trusted sources. Apache 2.0.
- Cost at scale
- The software is free; the cost is accelerators sized to the model's weights and KV cache. LMSYS sells no paid edition; enterprise support and consulting are arranged through sglang@lmsys.org.
- Limits
- About 6,000 open issues and pull requests; some new models arrive in vLLM first; we could not confirm fixes for the three CVEs Antiproof disclosed in May 2026.
What's good
- RadixAttention reuses shared prompt prefixes
- Wide hardware support beyond NVIDIA
- Common rollout engine for RL frameworks
- OpenAI-compatible server
What's not
- Several critical 2026 CVEs and slow disclosure responses
- Smaller ecosystem than vLLM
- Tuning assumes serving experience
- About 6,000 open issues and pull requests
Best for
Teams serving open models with heavy prompt reuse, such as agents and multi-turn chat.
Alternatives
SGLang: common questions
- Is SGLang worth it?
- SGLang is built for the agent and RL workloads that resend the same long prompts, and it belongs behind a locked-down network. We rate it 4.4 out of 5.
- How much does SGLang cost?
- Free and open source under the Apache 2.0 license.
- What are the downsides of SGLang?
- Several critical 2026 CVEs and slow disclosure responses; smaller ecosystem than vLLM; tuning assumes serving experience; about 6,000 open issues and pull requests.
- Who is SGLang best for?
- Teams serving open models with heavy prompt reuse, such as agents and multi-turn chat.
- How much does SGLang cost for a team or in production?
- The software is free; the cost is accelerators sized to the model's weights and KV cache. LMSYS sells no paid edition; enterprise support and consulting are arranged through sglang@lmsys.org.
- Is SGLang safe for company data?
- Runs entirely on your hardware with no hosted service. Several 2026 CVEs allow remote code execution when HTTP or ZMQ ports are reachable or untrusted model files are loaded, so bind to a private network, add authentication and load models only from trusted sources. Apache 2.0.
- What are the best SGLang alternatives?
- The closest alternatives we have reviewed are vLLM and TensorRT-LLM.
Answers checked Oct 23, 2026.
Make SGLang? Show this rating on your site
