Safety · Concept
Agent Safety as Runtime Contract
Position that agent safety must be harness-enforced at runtime via preventive gates plus evidential proof of good actions, not only training-time alignment.
Forecast horizon
Agent platforms will standardize dual-face runtime safety contracts and trajectory schemas.
Connections
Connections · 10
How this node ties into the rest of the map, and the evidence behind each link.
to · enables 4/10
Verifiability Gap in Agentic FinTech Governance Runtime evidential contracts are a proposed way to close post-decision verifiability shortfalls for agents.
+4 growth from · supports 4/10
Vero: Repository-Level Formally Verified Code Generation Benchmark Repository-level formal proofs supply strong evidential artifacts that runtime contracts could require before accepting agent work.
+4 growth from · informs 4/10
Silent Updates: Post-Deployment Disclosure Gap Both stress post-training deployment controls and verifiable linkage between evaluated and served agent/model behavior.
+4 growth from · informs 4/10
LLM-Assisted Dynamic Threat Analysis for AV Weaknesses Dynamic confirmation of attacker-reachable AV weaknesses shows why agent/tool stacks need runtime gates beyond static analysis.
+4 growth from · requires 4/10
LLM-Assisted Dynamic Threat Analysis for AV Weaknesses Executable harnesses and sanitizer-backed dynamic confirmation exemplify runtime evidence needs for safety-critical agents.
+4 growth from · informs 3/10
Rules vs Character Scaling Laws for AI Safety Design Both analyze complementary training-time character shaping versus inference/runtime rule enforcement for safety.
+3 growth from · requires 3/10
Verifiability Gap in Agentic FinTech Governance FinTech agent study finds reproducibility/controls often provider-owned, motivating runtime verification contracts.
+3 growth from · frames 3/10
Rules vs Character Scaling Laws for AI Safety Design Scaling-laws model treats rule enforcement filters as the inference-time complement to training-time character shaping.
+3 growth to · informs 3/10
Verifiability Gap in Agentic FinTech Governance Runtime preventive and evidential contracts are proposed responses to weak post-decision verifiability in agent systems.
+3 growth from · enables 3/10
Interactive PCP Verification of Probabilistic Claim Consistency Verifiable consistency of probabilistic safety claims can underpin honesty requirements in runtime safety contracts.
+3 growthSignal sources
Signal sources
Dated facts from primary sources in this direction.
US evaluation centre Jun 2025
In June 2025 the US AI Safety Institute was renamed the Center for AI Standards and Innovation (CAISI), pivoting toward security, standards and adversary-model assessment.
NIST → Frontier safeguards May 2025
Anthropic activated its ASL-3 deployment and security standard with Claude Opus 4 on 22 May 2025 — the first real-world trigger of a responsible-scaling tier, focused on blocking bio-weapon uplift.
Anthropic → Cross-border testing 2025
The International Network of AI Safety Institutes (launched Nov 2024) ran a third joint testing exercise focused on agentic AI systems across cyber and fraud strands.
European Commission — AI Office →