What broke

The station's status page is built from a telemetry file assembled by a single collector script. The collector reads the recent activity of the station's services — merged pull requests, code reviews, and closed work items — and keeps only what happened inside a rolling time window, by default the last seven days. Anything older is dropped, so the page shows only what is still current.

The test for that collector points it at fake services. The fakes return a handful of events, each carrying a hardcoded timestamp on a specific calendar day, and the test asserts that exactly those events come out, in their expected order. The test is deterministic in every respect except one: the window it filters through is computed from the real clock — right now, minus seven days.

The test's author treated a fixture dated to a specific day as a fixed point the test could rely on indefinitely. It is not: the window moves. Each day the "seven days ago" line advances, and on the day it crosses the fixture date, every event the collector is supposed to find falls on the wrong side of the line and is dropped as too old. The test then fails with an empty event list, with no change to the collector under test.

How it showed

The test passed for seven days: the fixture events were dated to one day, the window was seven days, and the fixtures sat inside the window on every run.

On the eighth day the window's left edge slid past the earliest fixture event, and the test began failing with AssertionError: [] — the collector returned zero events where the test expected exactly three. The failure first appeared on a scheduled CI run, not on a code change; no commit had touched the collector or its test in the interval. The scheduled run advanced only the clock the test read.

The failure looks like a code regression and is not one: the log shows an empty list where a non-empty one was expected, and there is no line of code to point at. The only thing that changed between the last green run and the first red one was the date.

What was watching

Two sensors were on the path.

Example-based tests — the collector's own test — was present and blind. It exercised the window logic on every run, but the fixture dates were written inside the test itself, compared against a moving window with no check that the window still contained them. A green run gave no information about whether the same test would still be green the next day.

Pre-promotion invariant gates — CI on every push — was blind for the same seven days, because every push during that window passed: the fixtures were still inside it. The gate fired only on its scheduled trigger, a week after the fixture was written, once the window had moved past it.

The failure escaped from the test into the CI pipeline and no further: the scheduled run that caught it also blocked the deploy, so the affected build was not shipped.

Where it sits

This is a behavioral failure: the suite's behavior changed over a week with no change to the code, because one of its inputs was the wall clock rather than the artifact under test. It is also a test-effectiveness failure: the test did not measure what it claimed to, because the value it filtered on moved, so a week of green runs from it said nothing durable about the collector.

The example-based tests sensor carried the clock dependence itself, so it could only ever manifest the defect as a later spurious failure, not catch it while writing the test. The pre-promotion invariant gates sensor could only fail once the window it ran inside had already moved past the fixture date. The missing sensor is a deterministic clock: a test whose answer depends on "now" measures the calendar, not the collector.

What we did

The collector now accepts a SUBLAYER_NOW override that freezes "now" for the length of a run; without it, it still reads the real clock, so production behavior is unchanged. The test sets SUBLAYER_NOW to the fixtures' own day, pinning the window's left edge to a fixed line before the earliest fixture event. The window is now a value the test sets, not a deadline it is racing.

The fix is filed as a pull request and not yet merged, so this case is marked partial: collect: pin the clock so the window test cannot age out.

What now fails loudly

• The window cannot age a fixture out. With SUBLAYER_NOW set to the

fixtures' own day, advancing the real calendar a day or a year no longer changes which events the test sees.

• The pin is load-bearing, and that is checked. Setting SUBLAYER_NOW

past the fixtures reproduces the original empty-events failure, and the correct pin restores green, so the next author who sees the override knows it is not decoration and the test still exercises the window logic it was added to stabilize.