Anthropic's Claude Automates the Proof of Code Safety

Anthropic’s AI model, Claude, has successfully generated approximately 180 million lines of mathematical proof code to verify the correctness of existing software libraries. This significant achievement, accomplished over a matter of weeks in collaboration with partner Theorem, marks a pivotal shift in how developers might approach software reliability.

The initiative addresses a long-standing challenge in the tech industry: proving that software operates without errors across all possible scenarios has historically been prohibitively expensive. Previously, such rigorous formal verification was reserved for high-stakes environments, such as the avionics systems of Airbus aircraft. For instance, a team of specialists required twenty person-years to formally verify the seL4 operating system kernel. By leveraging Claude, Anthropic’s Frontier Red Team has drastically reduced the time and human expertise needed for this complex task.

Rather than creating new software from scratch, the model focused on verifying existing code from 123 open-source libraries within Google’s OSS-Fuzz security project. The process involved translating crash tests into mathematical problems and analyzing compiled machine code. An independent verification tool named Rocq automatically checks the generated proofs. According to Anthropic, this workflow requires minimal human oversight, leading researchers to describe the process as fully autonomous, although the underlying formal foundations and tools were developed by human engineers at Theorem.

One of the most consequential outcomes of this automation is the discovery of hundreds of bugs that conventional testing methods had failed to detect for years. Many of these defects only manifested under highly specific conditions, such as unusually large input files or precise sequences of data access. During the verification of a widely used PDF reader, the team identified as many security vulnerabilities in a single week as are typically reported for that software over an entire year. In another instance, Claude uncovered a logical error that had remained undetected for more than a decade, demonstrating the model's ability to find latent issues even when a complete proof is not achieved.

Despite these successes, the project comes with strict practical limitations. Anthropic clarifies that the label 'formally verified' does not equate to being entirely error-free. The mathematical proofs hold true only under specific constraints, including a particular processor architecture, single-threaded execution, and input sizes under 64 megabytes. In real-world applications, these libraries often run on diverse hardware and configurations, meaning the verification does not cover every possible scenario.

The computational cost of this endeavor is substantial, with individual proofs consuming hundreds of millions of tokens. While Anthropic has not disclosed the total financial cost of the project, the efficiency gains are evident. A smaller test run indicated that proofs are reusable and that newer model iterations are becoming more efficient. Specifically, the Opus 5.5 model solved half of the proofs using approximately 40 percent fewer tokens than the previous Opus 5 version, suggesting a trajectory toward greater economic viability.

This development strengthens the argument for broader software automation. If AI systems can write code and subsequently guarantee its correctness within defined parameters, the focus of the industry may shift from asking if AI can program to asking if it can prove that its code is safe. Anthropic envisions a future where formal verification using AI becomes as routine and standard as current testing practices. Theorem’s roadmap already targets critical infrastructure components, including glibc, OpenSSL, and cURL, signaling an intent to apply this technology to the foundational layers of the internet's software stack.