Capabilities · Concept

LLM-Automated Scientific Reproducibility Assessment

LLM pipeline recovers original effect sizes in 41% of social science studies and reaches same qualitative conclusions in 96% of cases, outperforming human reanalysts on qualitative agreement.

Trend strength 4/10

Momentum +4/q

Confidence low

Status new

Forecast horizon

LLM-based reproducibility tools could scale scientific audit capacity significantly, reshaping research integrity workflows.

Connections

Connections · 1

How this node ties into the rest of the map, and the evidence behind each link.

to · enables 3/10

AI Agents Reshaping Knowledge Work

LLM-automated reproducibility assessment demonstrates AI agents taking over knowledge-intensive scientific audit tasks.

+3 growth

Signal sources

Dated facts from primary sources in this direction.

Task horizon doubling Mar 2025

The length of software tasks AI agents can do autonomously at 50% reliability has doubled about every 7 months — and since 2024 closer to every ~3 months.

METR →

Benchmarks saturating Apr 2025

In one year scores rose by 18.8, 48.9 and 67.3 points on MMMU, GPQA and SWE-bench; real-world software solve rate jumped from 4.4% to 71.7%.

Stanford HAI — AI Index 2025 →

Autonomous coding 2025–2026

On SWE-bench Verified (500 real GitHub issues), autonomous coding agents reached ~80–86% by late 2025, up from under 50% in early 2025.

Epoch AI →