ACI-046: The Seam That Takes Model Output

Thesis

Put each guarantee about model output where the model cannot get past it. Whatever a machine can check belongs in a schema or in code at the seam: a clamp with one home, a parse that fails closed, and metadata that never claims a check that did not run. The prompt keeps what a schema cannot express, and a tool's description says when not to use the tool.

Story

In April 2026 Lamplight, an Elixir platform for interactive fiction voiced by a language model, had a model extracting story entities whose fields must come from fixed vocabularies, and it kept returning values outside them. The first fix added a normaliser and listed the allowed values in the system prompt. The prompt half, 3.7KB, made the model "produce conservative, skeletal entities", dropping names, descriptions and voices, and was backed out the same day (Lamplight@4f7ce9e01, 2026-04-18). Two days later two more prompt variants went in and came out again, 6 and 13 minutes after landing, with no measured failure recorded for either; the later one never ran against the live model. What landed that evening was one tool per detected violation, its schema declaring the allowed values, so the provider rejects an out-of-vocabulary call (Lamplight@3245863cc, 2026-04-20). Its commit: "schema enforcement is the guarantee, not the prompt". Freeform prose, merges and renames stayed prompt-driven. Two hours later the same rule made a field required on a second tool's schema (Lamplight@f36376822, 2026-04-20), the fix whose testing side is ACI-016: See It Go Red.