Capabilities · Concept
Vero: Repository-Level Formally Verified Code Generation Benchmark
First benchmark for joint implementation and proof synthesis across multi-module Lean 4 repositories from real codebases.
Forecast horizon
Connections
Connections · 4
How this node ties into the rest of the map, and the evidence behind each link.
to · supports 4/10
Agent Safety as Runtime Contract Repository-level formal proofs supply strong evidential artifacts that runtime contracts could require before accepting agent work.
+4 growth from · supports 4/10
CAPRI: Contract-Aware Proof Repair for Isabelle Contract-aware Isabelle repair complements verified-repo agent goals by separating proof acceptance from unauthorized edits.
+4 growth from · enables 4/10
Reasoning as Learnable Rule-Based Process Treating reasoning as a sound rule-based process aligns with repository-level formal proof obligations in Vero.
+4 growth from · tracked by 3/10
arXiv LLM Agents Submissions (Aug 2026 batch) Vero benchmark is listed among the day’s arXiv LLM agents preprints.
+3 growthSignal sources
Signal sources
Dated facts from primary sources in this direction.
Task horizon doubling Mar 2025
The length of software tasks AI agents can do autonomously at 50% reliability has doubled about every 7 months — and since 2024 closer to every ~3 months.
METR → Benchmarks saturating Apr 2025
In one year scores rose by 18.8, 48.9 and 67.3 points on MMMU, GPQA and SWE-bench; real-world software solve rate jumped from 4.4% to 71.7%.
Stanford HAI — AI Index 2025 → Autonomous coding 2025–2026
On SWE-bench Verified (500 real GitHub issues), autonomous coding agents reached ~80–86% by late 2025, up from under 50% in early 2025.
Epoch AI →