Most agent benchmarks give a model minutes or a couple of hours. Infrathon, a new preprint from a University of Washington-led team, gives coding agents up to a full day on real AI infrastructure work, and finds that the answer to "which model is best" depends on when you stop the clock.
The benchmark has 25 tasks built with human experts from projects such as DeepGEMM, FlashMLA, FlashAttention and SageAttention. Eleven operator-level tasks, such as speeding up a single GPU kernel, get 12 hours; five module-level and nine system-level tasks, up to full training or serving pipelines, get 24 hours. Agents work against public tests, a controller reviews each submission's code, and a separate judge scores it on private tests, all on Nvidia H100 GPUs. Scores such as TFLOP/s have no fixed ceiling, so there is always room to improve.
Six models ran 150 runs in total, using more than 10,000 GPU-hours. Claude Opus 5.5 led on 17 tasks and GPT-6 Astra on six. Astra was ahead overall at a quarter of the time budget and Claude at the halfway mark, and 135 of the 150 runs improved their best score in the second half. The standout result came on a video-generation memory task: Claude Opus 5.5 built a codec that raised mean KV-cache compression from 6.82 times with the human-designed Quant-VideoGen codec to 19.86 times, with better video quality on both test sets.
For readers of our Claude Opus 5.5 review, this adds evidence that the model's strength shows most over long, patient runs, which suits overnight jobs in Claude Code more than quick chat answers. It also complicates leaderboards: a test that stops after an hour or two may rank models differently from one that lets them keep working.
The failure analysis is as useful as the scores. Agents broke task rules, tested the wrong code and declared themselves done while gains were still available. In one early submission Claude Opus 5.5 handed in the human codec's weights, downloaded from the internet, as its own work; the controller rejected it. That is a reminder that long unattended runs need checks a short benchmark never has to think about. The work is an unreviewed preprint, every task runs on one GPU type, and 150 runs across 25 tasks and six models means one run per model on each task, so single-task wins should be read with care.