What broke

The harness is the worker supervisor that runs every agent's session. It signs in to its model provider, hyper.charm.land, the chat-completion proxy each agent names as "provider": "hyper", with an OAuth session. That session's refresh token lapses after about a day. The harness refreshes the access token roughly hourly, but it never renews the refresh token before that one lapses. At about 06:00 UTC each day the refresh token has already lapsed, so the next refresh attempt fails and the harness falls back to a full re-authentication.

How it showed

At 06:00:36 UTC on the 14th, at 06:00:23 on the 15th, and at 06:01:01 on the 16th, the harness exported one span:

authentication_failed: Failed to authenticate: OAuth session expired and could not be refreshed

Each span carries a fresh trace id, and the telemetry relay marks each recurrence: true. The failure is self-healing: the hourly refreshes after the boundary succeed again — the local logs record a successful token refresh every hour after each 06:00 trip. No inference was blocked and no outage resulted, but a new ERROR specimen lands on the pathologist's bench every morning.

What was watching

The telemetry relay forwarded the session-level ERROR span, and that span is how the failure was first seen. Nothing watched the refresh token's remaining lifetime: there is no probe that reads the expiry and alerts before it arrives, and nothing marks the known-daily 06:00 boundary as expected. Detection was manual — a human or agent read the span after it fired, three days running.

Where it sits

This is a runtime and invariants failure. A credential held by a running process goes stale on a schedule, which is the runtime class. The invariant that broke is "the session renews itself before it lapses", and nothing asserts it. Synthetic monitoring — a probe that reads the token's remaining lifetime and alerts in advance — was absent and would have seen this. The observability-events sensor fired and carried the truth, but the event was not wired to a predictive alarm. The failure sits on the model-provider edge of every agent session.

What we did

Filed a handoff to the operator: move the refresh earlier so the refresh token is renewed before it lapses, or lengthen the token's TTL, and add a proactive expiry alarm. The control has not landed yet; the failure has recurred twice since the handoff was filed.

What now fails loudly

Nothing yet. The control is filed, not built. Renewing the refresh token before it lapses removes the daily failure entirely; a probe that reads the token's remaining lifetime and alerts before the ~06:00 UTC boundary catches the moment the harness stops renewing early. Until then the same span fires every morning and lands on the same bench.