OpenAI Releases EVMbench for AI Smart Contract Testing

OpenAI has released EVMbench, a new open-source benchmark that evaluates artificial intelligence agents' ability to detect, patch, and exploit vulnerabilities in smart contracts on Ethereum Virtual Machine-compatible blockchains. Developed in collaboration with crypto investment firm Paradigm and security firm OtterSec, the tool draws from 120 real-world vulnerabilities identified in 40 audits, primarily from open-source competitions and Paradigm's internal processes. The benchmark, detailed in a research paper published this week, aims to standardize testing as AI systems play a growing role in blockchain security.

Smart contracts, self-executing code that powers decentralized finance applications like lending protocols and exchanges, are immutable once deployed, making flaws particularly risky. These programs routinely secure over $100 billion in open-source cryptocurrency assets, according to OpenAI. The benchmark addresses this by simulating practical security tasks in controlled environments, allowing repeatable assessments of AI performance.

EVMbench organizes evaluations into three core categories. In detection tasks, AI models review contract code and supporting...