What is AnyJev?
AnyJev, from researchers at Nokia in Sunnyvale with co-authors at Tencent Hunyuan and the Hong Kong Polytechnic University, turns an ordinary open language model into a Jev-style decision model. You give it a state and a typed question (a choice, a yes/no or a score) and it returns a probability over the options, read from the model's next-token distribution in one prefill rather than parsed from generated text. It is not affiliated with TypeSafe AI, which makes Jev.
There are two routes. The training-free Decider works on any open causal LM through Hugging Face transformers, vLLM, SGLang or llama.cpp; its L0 level averages out position bias by rotating the options and divides out the model's preference for certain labels, with no labels needed. The Tacit models (1.7B to 9B, built on Qwen) are self-distilled to give in one pass the answers they reach when reasoning, and can send a capped share of low-confidence decisions to full reasoning.
The method is described in the AnyJev Technical Report (arXiv 2610.00831, 30 September 2026). The pip package anyjev reached 0.3.0 on 6 October 2026; the code is Apache-2.0 and the Tacit weights carry their base models' Apache-2.0 licence.
Is AnyJev worth it?
AnyJev's useful insight is that the labels an open model scores already contain a decent decision model, once two biases are removed. On Qwen3-8B with the 20-way BANKING77 task (300 test items), the training-free L0 readout raised accuracy from 0.747 to 0.803, cut answers that flip when the options are reversed from 0.230 to 0.073, and, most usefully, raised the share of decisions confident enough to automate at 5% error or less from 7.7% to 46.3%. Those are the authors' figures, but every number in the README is read from committed JSON and checked by a test.
The stronger evidence is external. On the community Decision Index, whose maintainers run the private parts and measure latency themselves, L0 on an unmodified Gemma-4-31B scored 60.22, fifth of 149 entries on 12 October 2026, and the fine-tuned AnyJev Tacitus 31B scored 66.47, tied third on edition 0.4.1 on 21 October, with a median latency of 95 ms on one RTX PRO 6000. The smaller Tacit-9B reaches 0.823 on the 231-item public JevBench set in one pass and 0.887 when 15.2% of decisions escalate to reasoning.
The trade-offs are stated plainly in the limitations page. An escalated decision costs seconds, not milliseconds, adding about 567 generated tokens on JevBench for Tacit-9B. The letter readout handles at most 26 options, Tacit on vLLM reads labels from the top 20 log-probabilities and counts anything else as zero, and nothing has been tested inside an agent loop. A recent fix shows why that matters: the vLLM backend used to read a missing label as -30, producing a confident wrong answer with no error, until a user reported it.
Against Jev itself the case is about where your text goes rather than accuracy. Jev is hosted and cheap, but every input leaves your network; AnyJev keeps decisions on a GPU you control, with a gateway that listens on 127.0.0.1 and has no authentication of its own. Compared with Laya's small encoder models, it suits teams already serving a 1.7B to 31B model on vLLM. It is a young research project, with two main authors and 11 issues filed so far, so expect the API to move.
Where it earns its place
A support platform team routing 50,000 tickets a day across 20 queues
Send each ticket to a queue and auto-close only the confident ones
Wrap the Qwen3-8B you already serve in the Decider at L0 and pass the queue list as a choice question; send anything below your probability threshold to a person. On BANKING77 that readout made 46.3% of decisions safe to automate at 5% error, against 7.7% from raw scores, with no labels.
A bank's ML team that cannot send customer text to a hosted model
Replace calls to a hosted decision API with an in-house equivalent
Serve Gemma-4-31B in FP8 on one RTX PRO 6000-class GPU and wrap it with the Decider at L0, the setup that placed fifth of 149 on the Decision Index. Put the 127.0.0.1 gateway behind your own authenticated proxy, since it has none.
An agent developer with one workstation GPU
Gate tool calls with yes/no checks that carry real probabilities
Load Tacit-4B with `adaptive=True` so at most 20% of every 1,000 decisions escalate to reasoning. On bev-decision that lifted accuracy from 0.663 to 0.704 with 17.8% escalated, at about 187 extra generated tokens per decision.
Before you commit
- Setup
- `pip install anyjev` on Python 3.10+, with extras `[hf]`, `[client]` or `[vllm]`; `Tacit.from_pretrained("morriszjm/Tacit-9B")` downloads weights from Hugging Face. Tacit-9B and Tacit-2B need transformers 5 or newer, and `vllm serve` on Hopper needs the CUDA toolkit.
- Integrations
- Backends for Hugging Face transformers, vLLM (in-process or a running server), SGLang and llama.cpp GGUF models; MLX and Ollama are on the roadmap. `python -m anyjev.serve` exposes POST /v1/decide and GET /v1/stats.
- Data and privacy
- Everything runs locally; weights come from Hugging Face. The gateway listens on 127.0.0.1 with no authentication. Code is Apache-2.0, Tacit weights inherit Apache-2.0 from Qwen, and datasets keep their own licences. No security advisories published.
- Cost at scale
- No paid edition. Tacit-9B runs on one GPU with transformers; on the Decision Index, Tacitus 31B ran on a single RTX PRO 6000 at a 95 ms median and 1,000 ms p95 latency. Escalation adds hundreds of generated tokens to each escalated decision, so the cap sets the bill.
- Limits
- At most 26 options; Tacit's one-pass readout is not averaged over option order; L0's batch prior hurts accuracy when one label dominates; no agent-loop evaluation yet; 7 open issues and two main authors.
What's good
- Works training-free on any open LLM through transformers, vLLM, SGLang or llama.cpp
- Removes order and label bias without labelled data
- Leaderboard entries measured by the Decision Index maintainers
- Limitations and every README number are documented and tested
What's not
- Escalated decisions take seconds and hundreds of extra tokens
- At most 26 options in the letter readout
- HTTP gateway has no authentication
- Not yet evaluated inside agent loops; small contributor base
Best for
ML engineers who already serve an open model on vLLM and want calibrated classification and routing decisions without training a separate classifier.
AnyJev alternatives
AnyJev: common questions
- Is AnyJev worth it?
- Careful research code that gets honest probabilities out of the open model you already run, with leaderboard results the board's maintainers measured themselves. We rate it 3.9 out of 5.
- How much does AnyJev cost?
- Free and open source under Apache-2.0, including the Tacit model weights. You supply the GPU.
- What are the downsides of AnyJev?
- Escalated decisions take seconds and hundreds of extra tokens; at most 26 options in the letter readout; hTTP gateway has no authentication; not yet evaluated inside agent loops; small contributor base.
- Who is AnyJev best for?
- ML engineers who already serve an open model on vLLM and want calibrated classification and routing decisions without training a separate classifier.
- How much does AnyJev cost for a team or in production?
- No paid edition. Tacit-9B runs on one GPU with transformers; on the Decision Index, Tacitus 31B ran on a single RTX PRO 6000 at a 95 ms median and 1,000 ms p95 latency. Escalation adds hundreds of generated tokens to each escalated decision, so the cap sets the bill.
- Is AnyJev safe for company data?
- Everything runs locally; weights come from Hugging Face. The gateway listens on 127.0.0.1 with no authentication. Code is Apache-2.0, Tacit weights inherit Apache-2.0 from Qwen, and datasets keep their own licences. No security advisories published.
- What are the best AnyJev alternatives?
- The closest alternatives we have reviewed are Laya, vLLM and vLLM Semantic Router.
Answers checked Oct 28, 2026.