ACI-016: See It Go Red

Thesis

A passing check is evidence only if it could have failed. Before you trust a green run, ask whether each test would pass against a stub that returns the right shape, take the fix away and name the tests that go red, and make any clean result find a case you planted before you believe it.

Story

In April 2026 a component in Lamplight, an Elixir application whose characters are voiced by a language model, began accepting model output that named an entity and carried nothing else, so 61 rich rationales went in and 61 empty entities came out (Lamplight@f36376822, 2026-04-20). The test suite stayed green throughout. The commit that fixed it says why: every test used the well-formed input shape, so none of them could see a malformed one. "Tests passed, behaviour rotted."