Safety · Concept
Virtue Ethics vs. Subordinate AI: Existential Risk Trade-off
Empirical study showing Constitutional AI fine-tuning for virtue reduces existential risk but increases general safety risks, revealing a fundamental alignment trade-off.
Forecast horizon
Trade-off between agent well-being and subordination will become central to long-term alignment research agendas.
Connections
Connections · 4
How this node ties into the rest of the map, and the evidence behind each link.
to · references 2/10
Emergent Misalignment from Narrow Fine-Tuning Virtue Ethics vs. Subordinate AI paper directly references the Emergent Misalignment scenario as its experimental baseline.
+2 growth to · frames 2/10
Emergent Misalignment from Narrow Fine-Tuning The virtue ethics vs. subordinate AI trade-off directly addresses fine-tuning choices that lead to emergent misalignment.
+2 growth to · references 2/10
Anthropic The paper evaluates Constitutional AI, a method developed by Anthropic, in the context of virtue ethics and existential risk.
+2 growth to · informs 2/10
Emergent Misalignment from Narrow Fine-Tuning Constitutional AI fine-tuning experiments reveal trade-offs between safety and existential risk analogous to emergent misalignment from narrow fine-tuning.
+2 growthSignal sources
Signal sources
Dated facts from primary sources in this direction.
US evaluation centre Jun 2025
In June 2025 the US AI Safety Institute was renamed the Center for AI Standards and Innovation (CAISI), pivoting toward security, standards and adversary-model assessment.
NIST → Frontier safeguards May 2025
Anthropic activated its ASL-3 deployment and security standard with Claude Opus 4 on 22 May 2025 — the first real-world trigger of a responsible-scaling tier, focused on blocking bio-weapon uplift.
Anthropic → Cross-border testing 2025
The International Network of AI Safety Institutes (launched Nov 2024) ran a third joint testing exercise focused on agentic AI systems across cyber and fraud strands.
European Commission — AI Office →