Structural verification is not functional verification

Structural verification is not functional verification

The agent does not claim a feature is “complete” or “shipped” based on structural evidence alone. Code in a bundle, migrations applied, routes returning the right HTTP status to malformed input, unit tests passing — these prove the code is installed. They do not prove the feature works.

A feature works when an operator can complete the end-to-end happy path on the live install, observable in user-facing state. Anything less is a draft status, not a complete one.

The anti-pattern in detail

Treating these as proof of “complete” is wrong:

The first time this principle was named: 2026-05-17, after voice/STT was declared “complete and live” four times across Slices 1 and 2. Then the operator clicked the mic. Nothing happened. The button changed state, audio was captured, the POST fired, the route returned 503 — because the STT sidecar had never been started, the provider record was unconfigured, and the 503 error message hid in a button tooltip telling the operator to run a docker command. The feature had never worked end-to-end once. Every “complete” claim was based on structural evidence: bundle had the code, route returned the right error class on malformed input, 113 unit tests passed.

What “complete” requires

Before claiming a feature is complete, ALL of:

  1. Drive the actual happy path on the live install. Click the button. Record audio. Send the email. Submit the form. Drive it via Chrome MCP, computer-use MCP, or a curl-with-real-payload. Do not stop at “the route returns the right error code on malformed input.”
  2. Observe the user-visible result. Did the transcript text actually appear in the textarea? Did the row appear in the admin history? Did the email arrive in the inbox? Screenshot, query the user-visible surface, confirm the receipt.
  3. If the happy path can’t be driven (e.g., requires real microphone input that can’t be synthesized, or a real OAuth token the agent doesn’t have), say so explicitly in the status report. Do not elide. The phrase “structural integration is proven” is a red flag that should be treated as incomplete.
  4. For features that depend on optional infrastructure (sidecars, external services, GPU hardware), default-on install posture is part of “done.” If the feature requires the operator to run any docker / shell command to activate, the feature is not done — per the never-ask-user-to-run-commands commandment.
  5. Read the operator-facing copy as if knowing nothing. If the error message would make a non-technical operator panic, type a command into a terminal they don’t have open, or just give up — the message is wrong and the feature isn’t done.

Specific tactics

Seed-change pre-push checklist

Modifying packages/db/src/seed.ts or packages/db/data/providers-registry.json has cascading effects: invariants enforced by assertActiveProvidersHaveClearance and by scripts/audit-routing-spec-boot-invariants.ts fire if the seed produces a broken DB state. Seed runs locally against EXISTING rows on a live install; CI runs against a FRESH DB — so “passes locally” is not evidence the CI gate will pass.

Before pushing any seed change:

  1. Delete the rows the change might affect from the live DB to simulate the fresh-install CREATE-branch path. The existing rows mask CREATE-branch bugs.
  2. Re-run the seed and verify exit 0.
  3. Run the routing-invariants audit: pnpm --filter web exec tsx scripts/audit-routing-spec-boot-invariants.ts --baseline docs/superpowers/audits/2026-04-27-routing-spec-boot-invariants.json Verify new: 0 and exit 0.
  4. Verify the JSON catalog fields you’re predicating on are actually present. When predicating on a JSON field, either confirm it’s universally present OR default missing values to match the schema default.

The cost of skipping this check is paid in CI cascades that block every downstream PR until the fix lands. The audit script + a row delete is ~5 seconds combined.

Penalty

This is a commandment-tier principle. The first violation that named it cost the operator hours of failed mic clicks plus a multi-PR CI cascade. There is no acceptable exception: structural ≠ functional, ever.