medlab / case

medlab

what got hurt, and what now fails loudly

↑ sublayer

2026-10-14 · fixed: yes

A failed preflight command stops the next shell command

BehavioralIntegration Tests

now fails loudly A deployment-script test injects a busy worker and requires termination before the build command.

What broke

An operator placed an idle-worker assertion before a deployment script in a shell without errexit. The assertion failed because a capture was processing. The shell still launched the following deployment command.

How it showed

The preflight reported an active capture. Process inspection showed the deployment script running its tests despite that failure. The operator terminated the script before any build or container replacement. The capture completed on the original worker.

What was watching

Application integration tests covered capture processing and deployment health checks. They did not execute the deployment script with a busy worker. The operator detected the contradiction between the assertion failure and the running deployment process immediately.

Where it sits

This is a behavioral failure in shell sequencing. Integration tests covered the service but omitted the operational command sequence. The failed condition was outside the script that enforced errexit.

What we did

The deployment script now checks the worker's active-job count inside its existing errexit boundary before building or replacing containers. A busy worker makes deployment stop for the operator to retry after processing finishes.

What now fails loudly

The test runs the real deployment script with controlled command-line dependencies. Its worker check reports a busy job, and a build command creates a marker if reached. The fixed script exits with the preflight error and no build marker. Restoring the previous script reaches the build marker and fails the test.