Safety · Concept
Forecasting Adversarial Capture in LLM Agent Populations
Individually calibrated agents can still be swayed by committed minorities; population response is forecastable from benign interaction logs.
Forecast horizon
Shifts safety unit of analysis from single-model audits to population dynamics.
Connections
Connections · 3
How this node ties into the rest of the map, and the evidence behind each link.
to · informs 6/10
Risks of Mass AI Agent Interactions Population-level adversarial capture shows multi-agent interaction can move decisions beyond single-agent audits.
+6 growth to · tracked by 3/10
arXiv AI Safety Feed Posted on the arXiv AI safety list as work on adversarial capture in LLM agent populations.
+3 growth to · drives 3/10
Risks of Mass AI Agent Interactions Committed-minority capture shows population-level failure modes invisible to single-agent audits.
+3 growthSignal sources
Signal sources
Dated facts from primary sources in this direction.
US evaluation centre Jun 2025
In June 2025 the US AI Safety Institute was renamed the Center for AI Standards and Innovation (CAISI), pivoting toward security, standards and adversary-model assessment.
NIST → Frontier safeguards May 2025
Anthropic activated its ASL-3 deployment and security standard with Claude Opus 4 on 22 May 2025 — the first real-world trigger of a responsible-scaling tier, focused on blocking bio-weapon uplift.
Anthropic → Cross-border testing 2025
The International Network of AI Safety Institutes (launched Nov 2024) ran a third joint testing exercise focused on agentic AI systems across cyber and fraud strands.
European Commission — AI Office →