What is billion-context?
billion-context is a local HTTP proxy, run as the bili command, that sits between a coding agent and its model API. It adds four tools to every conversation: compress folds a range of old messages into a summary, decompress brings the original back, search_context searches the folded history and acp_status reports how full the window is. The model decides when to call them, and the proxy rewrites what it sends upstream so folded ranges stay folded. The aim is one session that runs for days without hitting the context limit, while keeping the start of the prompt stable so the provider's prompt cache still applies.
It connects to roughly two dozen clients, including Claude Code, Codex, OpenCode, pi, Gemini CLI, Aider, Kimi Code and Copilot CLI, in one of three ways: a native plugin inside the client, a launcher (bili claude, bili codex) that starts the proxy and the client together, or a base URL pointed at http://localhost:8787/bili/. Clients that cannot change their base URL are handled by certificate interception: the launcher points them at a local certificate authority so the proxy can decrypt their TLS traffic to known model hosts.
It is written in TypeScript, installed from npm, and maintained mostly by one developer, ranxianglei, who has made 3,885 of its commits. The licence is MIT with an added term: any product people can see that uses it must credit billion-context with a link on its home page, documentation or credits page.
Is billion-context worth it?
billion-context attacks a real cost. Long agent sessions fill the context window, the agent's own summariser throws detail away, and every token is billed again on the next turn. Here the model folds old turns itself, in small ranges, and can unfold them when it needs the exact text. Because only the tail of the conversation changes, the provider's prefix cache keeps working: the README says a healthy session holds a 95 to 97% cache hit rate.
The evidence is more than most projects in this category offer, but it comes from inside the project. The author's paper reports 4.5 months of use across three machines, 174,327 model calls and 18.76B input tokens without overflowing a 204,800-token window. A 1,500-item evaluation labelled independent, committed to the repository on 31 October 2026, has billion-context scoring 79.20% on LongMemEval-S against 83.20% for the full context and 62.60% for the best retrieval setup, at $2.30 against $7.97. We could not confirm who ran it or that they are unconnected to the project.
The security picture needs reading in full before installing. Every prompt, file the agent reads and API key passes through the proxy. It has no authentication; it binds to 127.0.0.1 by default, but bili --host 0.0.0.0 and the Docker image listen on the network, and the README's own advice is to do that only on a trusted network. For clients without a base URL setting, the launcher points the client at the proxy's own certificate authority so it can read encrypted traffic to whitelisted model hosts. Debug mode logs client headers, and session state is kept on disk under ~/.local/share/billion-context.
It also changes itself. The proxy checks npm at start-up and every 3 minutes and installs newer versions in place, staggered at a random point between 15 minutes and 48 hours after release; the running process picks them up on restart. A separate advisory channel, polled on the same cadence even with auto-update off, can force-install a named version, including a rollback, unless advisoryCheck is set to false. That is a reasonable design for fixing a broken release quickly, and a supply-chain exposure for anyone whose prompts contain secrets. The project has shipped versions up to 0.1.213 since August 2026, a pace that makes pinning sensible.
The narrower alternative is to let the client compact on its own and keep traffic direct; context-mode and rtk shrink what goes into the window without sitting on the wire. billion-context goes further and works with more clients, but it belongs on a personal machine with auto-update and advisories turned off, not on a shared server or near production credentials.
Where it earns its place
A solo developer running one Claude Code session on a large refactor for several days
Keep the session alive without restarting and re-explaining the codebase
Run bili claude so the proxy starts with the client; old turns are folded as the window fills and reopened with decompress when exact text is needed. Turn off auto-update and the advisory check, and watch the cache hit rate with /acp-cache to see whether token costs actually fall.
A researcher comparing memory methods for long-lived agents
Test model-driven compression against plain retrieval on public benchmarks
The repository ships the 1,500-item LongMemEval-S, ContextBench and MemoryAgentBench evaluation, run on DeepSeek V4.1 Flash with v0.1.202. Rerunning it is the way to check the 79.20% and 0.3097 F1 figures before relying on them.
Before you commit
- Setup
- npm install -g billion-context --prefix=~/.local, then one of: bili plugin install <client>, bili <client> as a launcher, or a base URL of http://localhost:8787/bili/<upstream>. Needs Node 22 or newer for its SQLite storage; a Dockerfile is included.
- Integrations
- Claude Code, Codex, OpenCode, pi, omp, Gemini CLI, Qwen Code, Kimi Code, Aider, Copilot CLI, Amp, Kiro, Crush, Zed, goose, Hermes and others, over the Anthropic Messages, OpenAI Chat Completions and Responses wire formats. Can sit in front of another local gateway through the /bili/ prefix.
- Data and privacy
- All prompts and credentials pass through the proxy; it stores a 12-character hash of the key, not the key. Logs go to ~/.local/state/billion-context/bili.log and session state to ~/.local/share/billion-context. No authentication; binds to 127.0.0.1 unless --host is set, while the Docker image listens on 0.0.0.0:8787. Checks npm for updates and an advisory package every 3 minutes, and fetches announcements from the GitHub repository.
- Cost at scale
- The software is free; the saving depends on your sessions. In the project's evaluation, independent-session cost fell from $7.97 to $2.30 on LongMemEval-S and by 58.3% on ContextBench, with DeepSeek V4.1 Flash priced at $0.15 per 1M input tokens.
- Limits
- Summaries lose detail, and accuracy on LongMemEval-S was 4 points below the full context. Frequent releases, 1,081 open issues and pull requests, and one main maintainer. Plugin and launcher modes must not both be active, or compression runs twice.
What’s good
- The model folds and unfolds old turns itself instead of losing them to truncation
- Keeps the prompt prefix stable, so provider caching keeps working
- Works with about two dozen agents through plugins, launchers or a base URL
- Published long-running usage data and a 1,500-item evaluation
What’s not
- Every prompt and API key passes through it, with no authentication
- Auto-updates from npm every 3 minutes by default; advisories can force a version
- Some clients need TLS interception through a local certificate authority
- Attribution term on top of MIT for anything users can see
Best for
Individual developers on their own machine who run very long coding-agent sessions and are prepared to pin versions and review what the proxy does.
billion-context alternatives
billion-context: common questions
- Is billion-context worth it?
- A clever answer to agents running out of context, delivered as a self-updating, unauthenticated proxy that sees every prompt and key you send, so it asks for more trust than most teams should give it. We rate it 3.4 out of 5.
- How much does billion-context cost?
- Free and open source (MIT with an added attribution term). You still pay your model provider for every token the proxy forwards.
- What are the downsides of billion-context?
- Every prompt and API key passes through it, with no authentication; auto-updates from npm every 3 minutes by default; advisories can force a version; some clients need TLS interception through a local certificate authority; attribution term on top of MIT for anything users can see.
- Who is billion-context best for?
- Individual developers on their own machine who run very long coding-agent sessions and are prepared to pin versions and review what the proxy does.
- How much does billion-context cost for a team or in production?
- The software is free; the saving depends on your sessions. In the project's evaluation, independent-session cost fell from $7.97 to $2.30 on LongMemEval-S and by 58.3% on ContextBench, with DeepSeek V4.1 Flash priced at $0.15 per 1M input tokens.
- Is billion-context safe for company data?
- All prompts and credentials pass through the proxy; it stores a 12-character hash of the key, not the key. Logs go to ~/.local/state/billion-context/bili.log and session state to ~/.local/share/billion-context. No authentication; binds to 127.0.0.1 unless --host is set, while the Docker image listens on 0.0.0.0:8787. Checks npm for updates and an advisory package every 3 minutes, and fetches announcements from the GitHub repository.
- What are the best billion-context alternatives?
- The closest alternatives we have reviewed are context-mode, rtk (Rust Token Killer) and Claude Code Router.
Answers checked Nov 13, 2026.