Agentic Coding

ACI-038: Controls That Can Fail

Thesis

A check nobody has seen fail is no evidence that it ran. Prove every guard in both directions, with a refuse arm that asserts its reason, and treat the mutation and your own measurements as instruments too: a positive control that fires proves nothing unless the verdict changes when the subject does. Two readings count as two only if they could have disagreed, which takes instruments that differ in kind, aimed at a subject you can name.

Story

Intent, a Rust command-line tool that keeps a project's rules and gates in its repository, ran one of its integration test files in CI under || echo, so the step exited 0 whatever the tests did (Intent@f8948cce2, 2026-08-15). It was deleted after a run showed the suite passing, so the swallow hid nothing on the day it went. The harder cases are guards that run and report, and still cannot fail for the reason they exist.