AI Evaluability Gap

Sparse published audits in the Global South instantiate the broader evaluability gap under rapid deployment.

The AI Evaluability Gap concept reframes AI governance as requiring evidentiary foundations, not just system properties.

Benchmark-validity audit failures reveal the AI evaluability gap, showing governance assurance evidence can be silently manufactured.

Proposed specification layer is positioned as connective tissue making evaluation, mediation, and escalation composable for oversight.

Taxonomy aims to standardize how trained-model modifications are described for documentation and governance.

Shared vocabulary for how models were adapted improves documentation, change tracking, and evaluability for governors.

Capability-based scenario rating is proposed because likelihood-based AI risk prediction is conceded to fail.

Methodology makes decision-maker understanding explicit so safety cases alone do not fake evaluability.

When evaluators disappear, post-hoc reconstruction assumptions fail, widening the evaluability/assurance gap.

Short model-retirement intervals and version lock-in undermine assumptions that consequential AI decisions remain re-testable.

When evaluators disappear, reconstructability of model-mediated decisions fails unless commitment and independent verifiability are designed in.

Aviation certification analysis reveals a structural gap in AI governance documents analogous to the AI evaluability gap.

Affective safety harms are cumulative and relational, making them poorly captured by existing evaluation frameworks.