Skip to content
ReviewRamble

The weekly brief

AI, reviewed, in your inbox.

  • New reviews of tools, models, MCP servers and repos, with the verdict up front
  • Re-checked prices and ratings when something changes
  • New analysis pieces as they are published

Models, agents, MCP servers and the tools actually worth your time.

One email a week. Unsubscribe with the link in any brief. How we use your email

AI news & research

Research ·

Claude and Theorem formally verify 239 entry points across 123 open-source libraries

Anthropic and Theorem used Claude to turn existing fuzz tests into specifications and prove that compiled RISC-V binaries for 239 entry points across 123 OSS-Fuzz libraries never crash on any input under 64 MB, fixing hundreds of bugs along the way. The proofs, mostly in Rocq, total about 180M lines and took a few weeks.

Formal verification, proving mathematically that code behaves as specified, has long been the gold standard for security and too expensive for almost everyone. Anthropic gives a vivid benchmark: the seL4 kernel took two person-years to write and 20 to verify. Working with the verification company Theorem, Anthropic now reports that Claude produced such proofs at a scale that would have been unthinkable a few years ago.

The trick was to avoid writing specifications by hand. The team took fuzz tests from Google's OSS-Fuzz programme, small entry points where any crash counts as a bug, and turned each into a formal claim: the compiled RISC-V binary never crashes, including on memory-safety errors caught by AddressSanitizer, for any input under 64 MB. Over a few weeks, Claude proved this for 239 entry points across 123 libraries, patching the bugs it found along the way. The proofs, mostly in the Rocq proof assistant, total about 180 million lines, with a typical proof around 600,000 lines for a binary with a median of 18,000 instructions. Nobody needs to read them; the Rocq checker confirms them.

The bugs are the immediate payoff. Several were ones a fuzzer would rarely hit, such as an out-of-bounds write in a decompressor that only appeared on inputs of about 20 MB with one specific layout. Separate experiments proved the Linux kernel's zlib against a formal version of RFC 1950, and checked a Rust interpreter for a subset of Python, which matched CPython on 99.95% of 1.7 million random programs. Cost is falling too: on 36 small tasks, Claude Opus 5.5 proved half with about 40% fewer tokens than Opus 5, and sharing proofs between agents saved 17%.

For maintainers, this is the strongest evidence yet that machine-checked guarantees could become routine for common libraries, the way fuzzing did. Theorem is offering to run the pipeline for other open-source projects and has published a Claude skill.

The authors are careful about the limits, and readers should be too. Verified is not the same as bug-free. The proofs cover single-threaded, statically linked RISC-V builds using musl, while most users run these libraries on x86 or ARM with glibc. AddressSanitizer misses some memory bugs, not every entry point was covered, and callers can still misuse a library. Proofs can also take hundreds of millions of tokens to produce, which is the real barrier to doing this for the whole ecosystem.

Our take: Formal verification has always been too expensive for most code, and this suggests the cost is falling fast. The authors are clear that verified does not mean bug-free: the proofs cover single-threaded static builds that differ from how the software usually runs.

Our reviews:Claude Opus 5.5

Source: Anthropic

More stories