Safety · Concept
Imbalanced Pretraining Curricula for Selective Fine-Tuning Safety
Imbalanced task ordering during pretraining disentangles neural circuits, improving the selectivity of later safety fine-tuning that suppresses misaligned behavior.
Forecast horizon
Curriculum design may become a key lever for AI safety alignment alongside RLHF and constitutional AI methods.
Connections
Connections · 2
How this node ties into the rest of the map, and the evidence behind each link.
to · constrains 5/10
Emergent Misalignment from Narrow Fine-Tuning Imbalanced pretraining curricula improve selectivity of refusal fine-tuning, potentially constraining emergent misalignment from narrow fine-tuning.
+5 growth to · informs 3/10
Emergent Misalignment from Narrow Fine-Tuning Imbalanced pretraining curricula are shown to improve the selectivity of fine-tuning that suppresses misaligned behaviors.
+3 growthSignal sources
Signal sources
Dated facts from primary sources in this direction.
US evaluation centre Jun 2025
In June 2025 the US AI Safety Institute was renamed the Center for AI Standards and Innovation (CAISI), pivoting toward security, standards and adversary-model assessment.
NIST → Frontier safeguards May 2025
Anthropic activated its ASL-3 deployment and security standard with Claude Opus 4 on 22 May 2025 — the first real-world trigger of a responsible-scaling tier, focused on blocking bio-weapon uplift.
Anthropic → Cross-border testing 2025
The International Network of AI Safety Institutes (launched Nov 2024) ran a third joint testing exercise focused on agentic AI systems across cyber and fraud strands.
European Commission — AI Office →