What broke

The fleet's chess project runs its test suite in CI on a runner that is a purpose-built container image, stocked only with the language toolchain the project is written in (Go), not a developer's fully stocked machine. The suite — "the shipped demos" — includes two tests that read workflow YAML files, and those two parse YAML with a Python library. That Python dependency was written into the demo source and never declared anywhere the runner could provision it. The CI workflow installed Go and nothing else.

How it showed

Run 1474, on the pull request that retired an old tracking system, failed in the suite step. The failure appears after the Go build, the Go tests, and eleven other demos, in the twelfth:

FAIL python3 has no yaml module; this demo cannot parse the workflows it gates

A second demo failed the same way. The message named the missing module and stopped; it did not say what to install. The build, vet, and Go unit tests on the same run were green.

What was watching

The demos are smoke tests; each carries its own two-line check that the YAML library is present before it parses anything. That check was the only thing that fired, and it fired twelve demos deep, about two minutes into the run, after the rest of the suite had already passed. Nothing asked up front whether the runner had the libraries the suite needs. The dependency graph was blind to the Python library: it existed only inside the demo source, never in a manifest, so the graph had no entry to compare against the runner.

Where it sits

This is a runtime and invariants failure. The missing library lived in the runner's runtime environment — a Python package, outside the Go toolchain the workflow set up. The smoke-tests sensor was present and fired, but late, because it only checks at the moment each demo runs. The dependency-graph sensor was absent for this dependency: with no entry in a manifest, the graph had nothing to check.

What we did

The author added a file that lists the suite's Python dependencies, and a CI step that installs Python, installs that file, and asserts the installed version before the suite runs (pull request #89, merged). Separately, the suite's local entry point now runs a preflight that reads the same file and exits in seconds, with the install command, when a listed dependency is missing (pull request #90).

What now fails loudly

The suite's preflight reads the declared Python dependencies before running any demo and fails when one is absent. A demo that needs an undeclared library can no longer surface twelve demos deep with a message that names the module but not the fix.