Safety · Trend
RLHF Shallow Alignment and Partisan Structure Preservation
Dual-stance evaluation reveals LLMs represent sycophantic and factual agreement in geometrically distinct subspaces that standard steering cannot differentially target.
Forecast horizon
Mechanistic alignment auditing will become a regulatory requirement as RLHF's limitations become better understood.
Connections
Connections · 3
How this node ties into the rest of the map, and the evidence behind each link.
to · informs 4/10
Emergent Misalignment from Narrow Fine-Tuning Both studies reveal that RLHF and fine-tuning produce surface-level behavioral changes without deep representational alignment.
+4 growth from · applies to 3/10
Inference-Time Alignment via Probabilistic Model Blending Inference-time alignment via BlendIn addresses limitations of RLHF-based alignment by providing quality-aware guidance at deployment time.
+3 growth from · informs 3/10
Sycophancy Reduction via Activation Steering Limits Dual-stance evaluation reveals fundamental limits of activation steering for sycophancy reduction, pointing to deeper alignment challenges.
+3 growthSignal sources
Signal sources
Dated facts from primary sources in this direction.
US evaluation centre Jun 2025
In June 2025 the US AI Safety Institute was renamed the Center for AI Standards and Innovation (CAISI), pivoting toward security, standards and adversary-model assessment.
NIST → Frontier safeguards May 2025
Anthropic activated its ASL-3 deployment and security standard with Claude Opus 4 on 22 May 2025 — the first real-world trigger of a responsible-scaling tier, focused on blocking bio-weapon uplift.
Anthropic → Cross-border testing 2025
The International Network of AI Safety Institutes (launched Nov 2024) ran a third joint testing exercise focused on agentic AI systems across cyber and fraud strands.
European Commission — AI Office →