What is Calibrain Harness?
Calibrain Harness connects a fast classification model to crypto perpetual futures and treats every answer as a forecast to be checked. On each closed bar it asks the model seven labellable questions per symbol, such as which of +1.0% or -0.6% is touched first in the next two hours, turns the probabilities into expected value in code, and places bracketed long or short orders on Bybit testnet when the gates allow. Every prediction is then scored against mainnet prices, whether it was traded or not.
Two models are supported: Kairos from DMind via Minara Cloud, and Jev from TypeSafe, switchable live. A Python 3.12 service serves a Next.js 16 terminal on 127.0.0.1:8800 with order book, model signals, evaluation (Brier and RPS skill, reliability, AUC, walk-forward validation) and a decision log holding every request and response.
The repository is MIT-licensed, was created on 10 November 2026 by a single developer, has about 365 GitHub stars and 78 forks, and is marked under development with no releases. It is research software, not financial advice.
Is Calibrain Harness worth it?
Start with money. As shipped, the harness cannot touch real funds: the Bybit host is hard-coded to api-testnet.bybit.com with no mainnet switch, it starts observe-only with order placement off until you turn on AUTO-TRADE, and the README tells you to create testnet keys with Contract orders and positions permission and no withdrawal permission. Anyone who edits that host and supplies live keys would be running unaudited code on a real account, and could lose money quickly; nothing here is financial advice, and the author says past testnet results say nothing reliable about future returns.
Those results are worth reading. Over 10.5 hours on 10 November 2026 the bot made 769 model decisions and 40 trades and lost 2,601.01 testnet USDT after fees, most of it from market orders filling 3 to 11% away on thin testnet books. Once the model was shown each instrument's base rates, it beat them on short-bracket, direction and big-move questions over 320 decisions, but strict mode took no trades in 3 hours 45 minutes because a 15 basis point round trip ate about 0.25R of every setup.
What makes the project useful is the discipline around those numbers. The model only answers questions that price data can label, code rather than the model decides the trade, every decision is scored, calibration weights blend the model with base rates so a useless model stops driving trades, and the validation screen counts only independent windows so overlapping two-hour labels do not inflate the evidence. The safety limits are real too: a kill switch, a 3% daily loss limit, a 5,000 USDT notional cap and a 3x ceiling on borrowed exposure.
It is eight days old, written by one person, and changes defaults between commits. Next to TradingAgents, which debates a stock into a rated opinion, this is narrower and more rigorous: it measures whether a model's probabilities are worth anything before anyone trusts them.
Where it earns its place
A researcher at a company selling a classification model
Show whether our model forecasts short-horizon crypto moves better than base rates
Run the harness observe-only on a few symbols for several days; the EVAL and VALID screens report Brier skill against base rates with 90% bootstrap intervals on independent windows, and the export writes a Markdown report. Cost is model API calls, about 180 an hour for seven symbols on 5-minute bars.
A crypto trader learning systematic methods
See how fees and slippage change a strategy that looks good on paper
Turn on AUTO-TRADE in exploration mode on Bybit testnet at the default 0.1% risk and compare each trade's reference price, fill, slippage and R on the TRADES screen. No real money is involved, and the edge-versus-cost view shows the break-even cost.
Before you commit
- Setup
- Clone, create a Python 3.12 virtual environment, install requirements, copy .env.example, build the Next.js UI with Node 20+, then run python -m ctb.server and open 127.0.0.1:8800. Keys can be pasted in the terminal's KEYS dialog.
- Integrations
- Kairos (DMind via Minara Cloud) or Jev (TypeSafe, pinned to jev-1.14.0); Bybit mainnet public data for signals and Bybit testnet for orders; news RSS, Fear and Greed and macro context. Other models need a ModelSpec added in code.
- Data and privacy
- Keys are read from a gitignored .env with owner-only permissions and are never logged or sent to the browser. Every request, answer, label, trade and run is kept in a local SQLite database. Market state goes to the model provider. The server binds to 127.0.0.1 and refuses cross-origin writes.
- Cost at scale
- The harness is free; model calls are the cost. The default budget is 120 requests an hour, mirror averaging doubles requests, and seven symbols on 1-minute bars need about 900 an hour. Kairos and Jev prices are not given in the README.
- Limits
- Bybit testnet only, with thin books and synthetic flow; Kairos allows up to 8 questions a request and Jev 64k tokens; strict mode may take no trades for hours; results so far cover one day and one market regime.
What’s good
- Testnet host hard-coded; starts observe-only
- Scores every forecast, traded or not, against mainnet prices
- Calibration against base rates and independent-window validation
- Publishes its losing results and the reasons for them
What’s not
- Eight days old, one developer, under active change
- Supports only Kairos and Jev, both paid APIs
- Testnet liquidity distorts P&L; one day of results
- Not audited; unsafe to adapt for live funds
Best for
Quant researchers and model builders who want to test whether a fast classifier's probabilities carry an edge on crypto perpetuals without risking real money.
Calibrain Harness alternatives
Calibrain Harness: common questions
- Is Calibrain Harness worth it?
- Calibrain Harness is an unusually honest way to ask whether a classifier has a trading edge, and its own numbers so far say the answer is no. We rate it 3.4 out of 5.
- How much does Calibrain Harness cost?
- Free and open source (MIT). Needs a Kairos (Minara Cloud) or Jev (TypeSafe) API key, billed by those providers, and free Bybit testnet keys.
- What are the downsides of Calibrain Harness?
- Eight days old, one developer, under active change; supports only Kairos and Jev, both paid APIs; testnet liquidity distorts P&L; one day of results; not audited; unsafe to adapt for live funds.
- Who is Calibrain Harness best for?
- Quant researchers and model builders who want to test whether a fast classifier's probabilities carry an edge on crypto perpetuals without risking real money.
- How much does Calibrain Harness cost for a team or in production?
- The harness is free; model calls are the cost. The default budget is 120 requests an hour, mirror averaging doubles requests, and seven symbols on 1-minute bars need about 900 an hour. Kairos and Jev prices are not given in the README.
- Is Calibrain Harness safe for company data?
- Keys are read from a gitignored .env with owner-only permissions and are never logged or sent to the browser. Every request, answer, label, trade and run is kept in a local SQLite database. Market state goes to the model provider. The server binds to 127.0.0.1 and refuses cross-origin writes.
- What are the best Calibrain Harness alternatives?
- The closest alternatives we have reviewed are TradingAgents, Vibe-Trading and FinRL.
Answers checked Nov 18, 2026.