Any service that uses several language models faces a quiet daily choice: which model should answer this request? Send everything to the most expensive model and you waste money on easy questions; send everything to the cheapest and quality suffers on hard ones. Routers try to decide per prompt. A preprint from Italy's National Research Council and the University of Pisa, prepared for the ACM Web Conference 2027, adds a twist: sometimes the best answer is to ask several models at once.
PISA is the single-model version. It estimates how capable each model is for a given prompt by placing the prompt softly across clusters of similar requests and combining that with a predictor trained for each model. PISA-E extends this to ensembles, choosing both which models to call and how many, then combining their answers by weighting similar responses according to each model's estimated ability. The key practical feature is a cost-quality dial that can be turned at inference time without retraining, so an operator can favour cheaper answers at peak load or better ones for premium users with the same system.
The team tested on four public routing benchmarks, LLMRouterBench Track-2, RouterBench, FusionBench and SPROUT, both on familiar task groups and on held-out ones. Among single-model routers, PISA had the best average balance of cost and performance in both settings and ranked first in six of the eight combinations. When tuned for quality, PISA-E beat the strongest ensemble methods by 2.09 percentage points on familiar tasks and 2.87 on unfamiliar ones. At cheaper settings it still matched or beat them while cutting average cost by about 36% and 25%.
For teams that already use a gateway such as OpenRouter to reach many models through one account, this is the logic that could sit on top: one endpoint, with the system choosing per request. Our OpenRouter review notes that its strength is breadth across models and providers; deciding which to use for each prompt is left largely to you, and that is the gap PISA targets.
The gains are modest and the tests are offline. All approaches were scored on model responses supplied by the benchmarks, not on live traffic, where prompts drift, prices change and latency matters as much as cost. Ensembles also multiply calls, so savings depend on how often the router picks one model. The paper gives no figures for the router's own overhead in the sections we read, and production users will want that before switching.