Skip to content
ReviewRamble

The weekly brief

AI, reviewed, in your inbox.

  • New reviews of tools, models, MCP servers and repos, with the verdict up front
  • Re-checked prices and ratings when something changes
  • New analysis pieces as they are published

Models, agents, MCP servers and the tools actually worth your time.

One email a week. Unsubscribe with the link in any brief. How we use your email

Calibrain Harness

Forward-testing rig that lets fast classification models trade crypto perpetuals on Bybit testnet and scores every forecast.

Reviewed by ReviewRamble editors · Updated

View on GitHub Open source

Share this page:XLinkedInFacebookRedditWhatsAppEmail

Our rating

3.4/5

Calibrain Harness is an unusually honest way to ask whether a classifier has a trading edge, and its own numbers so far say the answer is no.

What is Calibrain Harness?

Calibrain Harness connects a fast classification model to crypto perpetual futures and treats every answer as a forecast to be checked. On each closed bar it asks the model seven labellable questions per symbol, such as which of +1.0% or -0.6% is touched first in the next two hours, turns the probabilities into expected value in code, and places bracketed long or short orders on Bybit testnet when the gates allow. Every prediction is then scored against mainnet prices, whether it was traded or not.

Two models are supported: Kairos from DMind via Minara Cloud, and Jev from TypeSafe, switchable live. A Python 3.12 service serves a Next.js 16 terminal on 127.0.0.1:8800 with order book, model signals, evaluation (Brier and RPS skill, reliability, AUC, walk-forward validation) and a decision log holding every request and response.

The repository is MIT-licensed, was created on 10 November 2026 by a single developer, has about 365 GitHub stars and 78 forks, and is marked under development with no releases. It is research software, not financial advice.

Is Calibrain Harness worth it?

Start with money. As shipped, the harness cannot touch real funds: the Bybit host is hard-coded to api-testnet.bybit.com with no mainnet switch, it starts observe-only with order placement off until you turn on AUTO-TRADE, and the README tells you to create testnet keys with Contract orders and positions permission and no withdrawal permission. Anyone who edits that host and supplies live keys would be running unaudited code on a real account, and could lose money quickly; nothing here is financial advice, and the author says past testnet results say nothing reliable about future returns.

Those results are worth reading. Over 10.5 hours on 10 November 2026 the bot made 769 model decisions and 40 trades and lost 2,601.01 testnet USDT after fees, most of it from market orders filling 3 to 11% away on thin testnet books. Once the model was shown each instrument's base rates, it beat them on short-bracket, direction and big-move questions over 320 decisions, but strict mode took no trades in 3 hours 45 minutes because a 15 basis point round trip ate about 0.25R of every setup.

What makes the project useful is the discipline around those numbers. The model only answers questions that price data can label, code rather than the model decides the trade, every decision is scored, calibration weights blend the model with base rates so a useless model stops driving trades, and the validation screen counts only independent windows so overlapping two-hour labels do not inflate the evidence. The safety limits are real too: a kill switch, a 3% daily loss limit, a 5,000 USDT notional cap and a 3x ceiling on borrowed exposure.

It is eight days old, written by one person, and changes defaults between commits. Next to TradingAgents, which debates a stock into a rated opinion, this is narrower and more rigorous: it measures whether a model's probabilities are worth anything before anyone trusts them.

Where it earns its place

  • A researcher at a company selling a classification model

    Show whether our model forecasts short-horizon crypto moves better than base rates

    Run the harness observe-only on a few symbols for several days; the EVAL and VALID screens report Brier skill against base rates with 90% bootstrap intervals on independent windows, and the export writes a Markdown report. Cost is model API calls, about 180 an hour for seven symbols on 5-minute bars.

  • A crypto trader learning systematic methods

    See how fees and slippage change a strategy that looks good on paper

    Turn on AUTO-TRADE in exploration mode on Bybit testnet at the default 0.1% risk and compare each trade's reference price, fill, slippage and R on the TRADES screen. No real money is involved, and the edge-versus-cost view shows the break-even cost.

Before you commit

Setup
Clone, create a Python 3.12 virtual environment, install requirements, copy .env.example, build the Next.js UI with Node 20+, then run python -m ctb.server and open 127.0.0.1:8800. Keys can be pasted in the terminal's KEYS dialog.
Integrations
Kairos (DMind via Minara Cloud) or Jev (TypeSafe, pinned to jev-1.14.0); Bybit mainnet public data for signals and Bybit testnet for orders; news RSS, Fear and Greed and macro context. Other models need a ModelSpec added in code.
Data and privacy
Keys are read from a gitignored .env with owner-only permissions and are never logged or sent to the browser. Every request, answer, label, trade and run is kept in a local SQLite database. Market state goes to the model provider. The server binds to 127.0.0.1 and refuses cross-origin writes.
Cost at scale
The harness is free; model calls are the cost. The default budget is 120 requests an hour, mirror averaging doubles requests, and seven symbols on 1-minute bars need about 900 an hour. Kairos and Jev prices are not given in the README.
Limits
Bybit testnet only, with thin books and synthetic flow; Kairos allows up to 8 questions a request and Jev 64k tokens; strict mode may take no trades for hours; results so far cover one day and one market regime.

What’s good

  • Testnet host hard-coded; starts observe-only
  • Scores every forecast, traded or not, against mainnet prices
  • Calibration against base rates and independent-window validation
  • Publishes its losing results and the reasons for them

What’s not

  • Eight days old, one developer, under active change
  • Supports only Kairos and Jev, both paid APIs
  • Testnet liquidity distorts P&L; one day of results
  • Not audited; unsafe to adapt for live funds

Best for

Quant researchers and model builders who want to test whether a fast classifier's probabilities carry an edge on crypto perpetuals without risking real money.

Calibrain Harness alternatives

Calibrain Harness: common questions

Is Calibrain Harness worth it?
Calibrain Harness is an unusually honest way to ask whether a classifier has a trading edge, and its own numbers so far say the answer is no. We rate it 3.4 out of 5.
How much does Calibrain Harness cost?
Free and open source (MIT). Needs a Kairos (Minara Cloud) or Jev (TypeSafe) API key, billed by those providers, and free Bybit testnet keys.
What are the downsides of Calibrain Harness?
Eight days old, one developer, under active change; supports only Kairos and Jev, both paid APIs; testnet liquidity distorts P&L; one day of results; not audited; unsafe to adapt for live funds.
Who is Calibrain Harness best for?
Quant researchers and model builders who want to test whether a fast classifier's probabilities carry an edge on crypto perpetuals without risking real money.
How much does Calibrain Harness cost for a team or in production?
The harness is free; model calls are the cost. The default budget is 120 requests an hour, mirror averaging doubles requests, and seven symbols on 1-minute bars need about 900 an hour. Kairos and Jev prices are not given in the README.
Is Calibrain Harness safe for company data?
Keys are read from a gitignored .env with owner-only permissions and are never logged or sent to the browser. Every request, answer, label, trade and run is kept in a local SQLite database. Market state goes to the model provider. The server binds to 127.0.0.1 and refuses cross-origin writes.
What are the best Calibrain Harness alternatives?
The closest alternatives we have reviewed are TradingAgents, Vibe-Trading and FinRL.

Answers checked Nov 18, 2026.