Can Claude Prove What Fuzz Tests Find? Anthropic’s Work Explained

🔍 Read the full analysis: Can Claude Prove What Fuzz Tests Find? Anthropic’s Work Explained on ThorstenMeyerAI.com

Before you orderOffer from Amazon

Get hardware and tech essentials delivered free with Prime

• Fast, free delivery on millions of items
• Prime Video, Amazon Music and more included
• Member-only deals all year

Start your free Prime trial →Free trial for eligible customers · Cancel anytime

As an affiliate, we earn on qualifying purchases.Hide

Anthropic’s Frontier Red Team and collaborator Theorem said on October 22 that Claude generated machine-checked proofs for 239 fuzz-test entry points across 123 open-source libraries. The proofs establish that specified compiled programs do not crash on inputs under 64 MB, under stated assumptions; the effort also found bugs that the teams say years of fuzz testing had not uncovered.

The team used each existing fuzz test as the basis for a specification: for every input below the stated 64 MB limit, the test should not crash. The libraries and tests were compiled into instrumented RISC-V executables, with AddressSanitizer used to make certain memory errors trigger a crash. Claude then generated proofs in Rocq, a formal proof system, and Rocq’s checker verified them against the specifications.

Anthropic reports that the effort ran over a few weeks and produced about 180 million lines of Rocq proof. A typical proof was about 600,000 lines and covered a binary with a median length of 18,000 RISC-V instructions. The companies say the proofs can be mechanically checked without people reviewing every line; confidence rests instead on the theorem statements, formal models and proof checker, among other trusted components.

According to Anthropic, the process discovered hundreds of bugs. Examples included an out-of-bounds write requiring an unusual layout in an input of about 20 MB, and an image-decoder read error requiring a 64 MB input and more than 2 GB of memory. The report also describes a bug that required a sequence of calls involving two handles to the same file, and a logic error in a feature intended to restrict which files an application could load.

Proofs Reach Beyond Fuzzing

Fuzzing tests many generated or altered inputs, but it cannot guarantee that every reachable path has been exercised. A proof can establish a property across all inputs covered by its specification and model. That difference may help explain why the team’s proof attempts exposed cases that ordinary fuzzing had not reached, including unusually large inputs and specific sequences of operations.

The work matters because software security teams often use testing to find defects but cannot infer from a test run that a whole class of failures is absent. Formal verification can make stronger claims, though it has traditionally taken substantial specialist time. Anthropic cites the seL4 microkernel project as an example of the labor involved: two person-years to build and 20 person-years to verify. The reported Claude work suggests AI may reduce some of the effort to produce proofs, but the project’s token demands and narrow proof scope remain important constraints.

The result is not evidence that the libraries are generally safe or bug-free. It is evidence, as described by Anthropic and Theorem, that specified properties were proved for particular compiled programs under particular assumptions. That distinction matters to maintainers and users deciding how much weight to give the findings.

From Fuzz Tests to Rocq Proofs

A fuzz test supplies an entry point into a library and checks how the program behaves when fed many inputs. In the method described by Anthropic, an existing test becomes a formal specification rather than merely a source of test cases. Theorem’s models describe the relevant RISC-V instructions and Linux process behavior; Claude generates a Rocq proof, which a checker validates. When a proof attempt fails, that can indicate a bug in the program or a challenge in completing the proof. The team says Claude could patch code and attempt a proof again.

Anthropic describes the work, to its knowledge, as the first large-scale use of AI for formal verification of production software at the assembly-code level. It says it is publishing proofs that did not require patches, certain previously unpublished trusted components, and specifications for fuzz tests it could not prove. The source report also notes that proof generation could consume hundreds of millions of tokens, while newer models and sharing proof work may reduce the cost.

The claimed scope is specific: single-threaded, static RISC-V binaries linked against musl. It does not directly establish the same property for commonly used x86 or ARM builds linked against glibc. The proof concerns behavior detected by the modeled test and instrumentation, not every security or correctness property of the library.

“Proofs can take hundreds of millions of tokens.”

— Anthropic, on the proof-generation cost

Limits of the Verified Builds

The proofs do not cover every version, architecture or deployment of the libraries. In particular, the reported work concerns single-threaded static RISC-V binaries linked against musl, rather than the x86 and ARM builds with glibc that many users run. The 64 MB input bound and the assumptions embedded in the formal models also define what the no-crash result means.

“No crash” is not synonymous with “no bugs.” Some defects, including the file-restriction logic error described in the report, may not cause a crash. The supplied material does not specify how many of the hundreds of reported bugs were independently confirmed, how many fixes have been merged or released, or the final token and computing costs across all proofs. It also does not establish whether the approach will scale economically to a wider range of software.

Published Proofs and Follow-Up

Anthropic says it is releasing the proofs that did not require patching, trusted components that were not already public, and specifications for fuzz tests it could not prove. Those materials can let other researchers inspect the formal statements and checking process. The next practical milestones are whether library maintainers confirm and address the reported defects, and whether the proof artifacts can be reproduced and adapted to other builds.

Further reports would be needed to establish how often this method finds exploitable vulnerabilities, how much it costs in practice, and whether proofs can be extended to common x86 and ARM configurations. Until then, the reported results support a narrower conclusion: Claude can help generate machine-checkable proofs for defined properties in selected compiled programs, but the scope and expense of that work remain material limits.

Key Questions

What did Claude prove?

Anthropic and Theorem say Claude generated Rocq-checked proofs that 239 specified fuzz-test entry points in compiled programs would not crash on inputs under 64 MB, subject to the models and assumptions used.

Does this show the libraries are bug-free?

No. The proof covers a defined no-crash property for particular binaries and inputs. It does not rule out every defect, security issue or behavior outside the specification.

How did the process find bugs fuzzers missed?

Fuzzers sample inputs and execution paths. A successful proof has to account for all paths covered by its formal model, which can expose unusual input sizes or call sequences that testing did not reach.

Do the results apply to software people commonly run?

Not directly to every build. The reported proofs cover single-threaded static RISC-V binaries linked against musl; Anthropic says they do not establish the same result for x86 or ARM builds linked against glibc.

What happens after the report?

Anthropic says it is publishing selected proof artifacts, trusted components and specifications. Maintainer responses, independent reproduction and work on other architectures will help clarify the findings’ practical reach.

Source: ThorstenMeyerAI.com