Capabilities · Concept
LLM-Automated Scientific Reproducibility Assessment
CORE-Bench v1.1 demonstrates that benchmark saturation does not exhaust utility; agents can still be measured on efficiency, reliability, construct validity, and human-AI collaboration uplift.
Forecast horizon
LLM-based reproducibility tools could scale scientific audit capacity significantly, reshaping research integrity workflows.
Connections
Connections · 1
How this node ties into the rest of the map, and the evidence behind each link.
Signal sources
Signal sources
Dated facts from primary sources in this direction.
Task horizon doubling Mar 2025
The length of software tasks AI agents can do autonomously at 50% reliability has doubled about every 7 months — and since 2024 closer to every ~3 months.
METR → Benchmarks saturating Apr 2025
In one year scores rose by 18.8, 48.9 and 67.3 points on MMMU, GPQA and SWE-bench; real-world software solve rate jumped from 4.4% to 71.7%.
Stanford HAI — AI Index 2025 → Autonomous coding 2025–2026
On SWE-bench Verified (500 real GitHub issues), autonomous coding agents reached ~80–86% by late 2025, up from under 50% in early 2025.
Epoch AI →