What it is
This is the code side of Laya, Convai Innovations' small open decision models. The pip package (first released on 18 September 2026) loads three checkpoints, an English one and a typed-decisions one on ModernBERT-large (421M parameters) and a multilingual one on mmBERT-base (322M), and answers choice, score and yes/no questions in one forward pass. Its Router detects the script and language of each request in under a millisecond and sends it to the checkpoint that can read it.
Around that sit optional extras: laya-serve, an HTTP server that accepts TypeSafe Jev's POST /v1/systemone requests so Jev clients can switch by changing the base URL; an MCP server; LangChain, LangGraph, LlamaIndex and CrewAI integrations; ONNX and GPU fast paths; and TypeScript, Rust and .NET SDKs. The repository also holds fine-tuning notebooks for Kaggle's free T4 GPUs and Apple Silicon. Everything is Apache 2.0.
Our review
The repository is better than the models it serves. The Router solves a failure the authors found and documented: the English checkpoint scored 0.000 accuracy on Khmer while reporting 0.952 confidence, so confidence thresholds could not catch it, and the router now checks the script before the model runs. Speed is real, at 33 ms for one question and 7.2 ms a question in batches on a T4, per the README.
The server needs care. `laya-serve` binds to 0.0.0.0:8000 by default and requires a bearer key only when LAYA_API_KEY is set, so out of the box anyone on the network can call it. Set the key, or LAYA_HOST=127.0.0.1, before starting it anywhere shared. It speaks Jev's wire format, caps batches at 64 states and returns 422 errors for malformed questions.
The models are the weak part. On the project's own typed-decisions benchmark of 2,000 decisions, the base English checkpoint scores 0.362 accuracy and the fine-tuned typed-decisions checkpoint 0.766. The README also says both shipped checkpoints are over-confident and that the multilingual one has no fitted temperatures, so its confidence scores need calibrating on your data before you gate anything on them. The Kaggle notebook runs the whole fine-tuning loop on two free T4s.
The pace is striking: the repository was created on 18 September 2026, has about 34,000 stars and reached v0.4.13 on 21 October, three releases in four days. The more useful comparison is not with Jev but with a conventional fine-tuned classifier. Laya takes the questions as input, so a new decision needs a schema rather than a training run, until accuracy demands one anyway.
Where it earns its place
A support-operations engineer at a 30-agent help desk
Route each ticket by department, urgency and churn risk
Call Router.predict on every ticket for three typed answers in one pass. Fine-tune first on past tickets with the Kaggle notebook, which costs nothing on free T4s; the base checkpoint's 0.362 benchmark accuracy is too low to route on.
A team already calling TypeSafe Jev's hosted API
Self-host decisions to keep ticket text in-house
Run laya-serve and repoint the client's base URL. Set LAYA_API_KEY and bind to 127.0.0.1 or a private address; the trade is lower accuracy than a hosted model unless you fine-tune.
A product team with customers writing in dozens of languages
Triage messages without a separate model per language
The Router sends non-English text to laya-multilingual automatically. Long documents need max_len=8192; the README reports 16 to 18 of 20 correct up to about 4,000 tokens and more variable results beyond.
Before you commit
- Setup
- `python -m pip install laya` on Python 3.10+; the Router downloads a checkpoint from Hugging Face on first use. `laya[serve]` adds the HTTP server.
- Integrations
- Extras for serve, mcp, langchain, llamaindex, crewai, onnx and a TileLang GPU fast path; laya-ts on npm; Rust and .NET SDKs; community Haskell and PHP clients.
- Data and privacy
- Weights come from Hugging Face; inference runs locally. laya-serve binds 0.0.0.0:8000 and is unauthenticated unless LAYA_API_KEY is set. Code and weights are Apache 2.0; no security advisories published.
- Cost at scale
- No paid edition. At the README's 7.2 ms a question batched, one T4 handles roughly 140 questions a second; CPU inference works with LAYA_THREADS to cap threads.
- Limits
- Checkpoints are over-confident as shipped; long-document accuracy varies past about 4,000 tokens; releases arrive every few days, so pin a version; 93 open issues.
What's good
- Language router that avoids confident wrong answers on unreadable scripts
- Jev-compatible server, MCP server and framework integrations
- Fine-tuning notebooks for free Kaggle GPUs and Apple Silicon
- 33 ms per question on a T4, per the README
What's not
- Base English checkpoint scores 0.362 on its own benchmark
- Server binds 0.0.0.0 with no authentication unless a key is set
- Shipped checkpoints are over-confident until calibrated
- Fast-changing API; pin versions
Best for
Developers who will fine-tune a small decision model on their own data and want a ready router, server and training loop.
Alternatives
Laya: common questions
- Is Laya worth it?
- Thoughtful plumbing around small decision models that only become accurate once you fine-tune them on your own labels. We rate it 3.6 out of 5.
- How much does Laya cost?
- Free and open source (Apache 2.0), including the model weights.
- What are the downsides of Laya?
- Base English checkpoint scores 0.362 on its own benchmark; server binds 0.0.0.0 with no authentication unless a key is set; shipped checkpoints are over-confident until calibrated; fast-changing API; pin versions.
- Who is Laya best for?
- Developers who will fine-tune a small decision model on their own data and want a ready router, server and training loop.
- How much does Laya cost for a team or in production?
- No paid edition. At the README's 7.2 ms a question batched, one T4 handles roughly 140 questions a second; CPU inference works with LAYA_THREADS to cap threads.
- Is Laya safe for company data?
- Weights come from Hugging Face; inference runs locally. laya-serve binds 0.0.0.0:8000 and is unauthenticated unless LAYA_API_KEY is set. Code and weights are Apache 2.0; no security advisories published.
- What are the best Laya alternatives?
- The closest alternatives we have reviewed are colibri, vLLM and SGLang.
Answers checked Oct 23, 2026.