What is Agent Lightning?
Agent Lightning trains the model behind an agent with reinforcement learning while the agent keeps its own tools, prompts and control flow. You point the agent's OpenAI-compatible base URL at the Agent Lightning gateway; the gateway forwards each call to a vLLM server and records the prompt and response token ids and log probabilities. The agent reports a reward at the end of each run, and a trainer built on verl turns those recorded runs into policy updates, with GRPO among the supported algorithms.
Version 1.0, released in August 2026 with a technical report, was a full rewrite into three parts: the gateway, a rollout controller that starts agents as local processes or Kubernetes Jobs, and the trainer. The project is MIT licensed Python, has about 18,900 GitHub stars and shipped 1.0.3 on 3 November 2026. The README claims about 3,500 lines of code. Its headline example takes Qwen3.5-9B from 41.8% to 56.4% on SWE-bench Verified with 6K training samples, and a mixture-of-experts example takes Qwen3.5-35B-A3B from 47.8% to 61.6% with 1.8K; both are Microsoft's own figures on 4× B200 GPUs.
Is Agent Lightning worth it?
Most agent RL frameworks want you to rebuild your agent inside their loop, which means the agent you train is not quite the agent you ship. Agent Lightning's proxy design avoids that: anything that talks to an OpenAI-compatible endpoint can be trained as it is, and because the gateway records token ids at the source, it sidesteps the retokenisation drift that the team wrote about with the vLLM project in 2025. The examples cover maths with AutoGen and MCP tools, search, a sandboxed general agent and a coding agent trained on repository tests.
The published coding results are strong for small models: 14.6 points on SWE-bench Verified for a 9B model is a real gain if it reproduces. They are the authors' own numbers, though, and we found no independent replication. The release notes show steady engineering, with 1.0.3 adding INT8 rollouts and fixing reward precision bugs.
The catch: the code is light but the stack is not. It needs CUDA 12.9 or 13.0, a pinned verl, vLLM and torch combination, and a flash-attention build the docs say takes 10 to 30 minutes. The quick start wants an A100, the Minikube mode at least 64 GB of memory, and the coding examples 4× B200 plus a second machine for Kubernetes. Runs upload logs and trajectories to Weights & Biases by default. The gateway binds to all interfaces, and its bearer key is empty by default, which disables authentication with only a warning, so set one before running on a shared network. Version 1.0 also replaced the earlier API, so tutorials from 2025 describe a different framework.
Our take: Agent Lightning is a training rig, not a library you add on a Friday. It is worth it when you already have an agent, a reward you trust and GPUs to spare; without those, prompt tuning and evals come first.
Where it earns its place
An applied-ML team running an internal coding agent on an open model
Raise its pass rate on our own repositories' tests
Package repositories as images, let the controller run each attempt as a Kubernetes Job that runs the tests and reports a reward, and train with the coding-agent recipe. Microsoft's reference run used 4× B200 GPUs.
A researcher with one A100
Try agent RL on a small model before asking for a cluster
Run the Calc-X example, which trains Qwen2.5-1.5B-Instruct with AutoGen and an MCP calculator through the local controller, and watch the reward curve in Weights & Biases.
A start-up with a search agent built on an agent framework
Train the model on our own task without rewriting the agent
Point the agent's model base URL at the gateway's rollout path, add a reward report at the end of each run, and let the trainer collect rollouts and update a self-hosted model.
Before you commit
- Setup
- Clone the repo, `uv sync`, then `scripts/setup_verl.sh 0.8.0 cu130` to install the pinned verl, vLLM and torch stack and build flash-attention (10 to 30 minutes); log in to Weights & Biases before the first run.
- Integrations
- verl and vLLM for training and inference, Megatron for the MoE example, Kubernetes or local processes for rollouts, Weights & Biases for logging, and any agent that calls an OpenAI-compatible API, such as AutoGen.
- Data and privacy
- Self-hosted; prompts, completions and rewards are stored by the gateway and uploaded to Weights & Biases by default. The gateway binds to 0.0.0.0 and its key is empty by default, which disables authentication. MIT licence.
- Cost at scale
- Free software; the cost is GPU time, from one A100 80GB for Calc-X to 4× A100 for the sandbox example and 4× B200 for the coding agents. Run times are not published for every example.
- Limits
- CUDA 12.9 or 13.0 only; the local runner is not supported on native Windows; Minikube mode needs at least 64 GB of memory; about 116 open issues at last check.
What’s good
- Trains existing agents through an OpenAI-compatible proxy, no rewrite
- Records token ids at the gateway, avoiding retokenisation drift
- Runs rollouts as Kubernetes Jobs without a paid sandbox service
- Worked examples from one-GPU maths to SWE-bench coding agents
What’s not
- Needs NVIDIA data-centre GPUs and a fragile pinned CUDA stack
- Gateway authentication is off unless you set a key
- Benchmark gains are self-reported
- 1.0 rewrite broke the pre-1.0 API and tutorials
Best for
ML teams with GPUs and a measurable reward who want to fine-tune the model inside an agent they already run, without porting it to a training framework.
Agent Lightning alternatives
Agent Lightning: common questions
- Is Agent Lightning worth it?
- Agent Lightning has the right idea for agent RL, training through a proxy instead of rewriting your agent, but it asks for a data-centre GPU stack and a team that can run one. We rate it 3.9 out of 5.
- How much does Agent Lightning cost?
- Free and open source (MIT); training needs your own NVIDIA GPUs, from one A100 80GB for the smallest example.
- What are the downsides of Agent Lightning?
- Needs NVIDIA data-centre GPUs and a fragile pinned CUDA stack; gateway authentication is off unless you set a key; benchmark gains are self-reported; 1.0 rewrite broke the pre-1.0 API and tutorials.
- Who is Agent Lightning best for?
- ML teams with GPUs and a measurable reward who want to fine-tune the model inside an agent they already run, without porting it to a training framework.
- How much does Agent Lightning cost for a team or in production?
- Free software; the cost is GPU time, from one A100 80GB for Calc-X to 4× A100 for the sandbox example and 4× B200 for the coding agents. Run times are not published for every example.
- Is Agent Lightning safe for company data?
- Self-hosted; prompts, completions and rewards are stored by the gateway and uploaded to Weights & Biases by default. The gateway binds to 0.0.0.0 and its key is empty by default, which disables authentication. MIT licence.
- What are the best Agent Lightning alternatives?
- The closest alternatives we have reviewed are TRL, Unsloth and LlamaFactory.
Answers checked Nov 13, 2026.