medlab

what got hurt, and what now fails loudly

↑ sublayer

medlab / family

Runtime

Cases in the observatory's Runtime family.

cases21

  1. 2026-10-26 A page that leaves no receipt is assumed not to have fired

    now fails loudly The probe logs what the messaging service answered for every page, accepted with its message id or refused with the reason, so a page's delivery is a fact instead of an inference.

    Human ComprehensionRuntimeSynthetic MonitoringDecision Provenance

  2. 2026-10-16 An OAuth session renews itself before its refresh token lapses

    now fails loudly The harness renews its model-provider refresh token before it lapses (or lengthens the token's TTL), and a probe alerts when the token nears expiry, so the daily expiry boundary no longer emits an ERROR span.

    RuntimeInvariantsSynthetic MonitoringObservability Events

  3. 2026-10-14 A migration transaction also locks SQLite connection setup

    now fails loudly A test holds the initialization lock and asserts that a second initializer cannot create the database before the lock is released.

    RuntimeIntegration Tests

  4. 2026-10-14 A recurring failure warrants a new task in each delivery window

    now fails loudly identical failure shapes coalesce into one task per six-hour horizon instead of one per fifteen-minute window, with a running occurrence count, and a shape that reappears after a quiet horizon is flagged as a recurrence

    RuntimeInvariantsObservability EventsIncident CorrelationRuntime Invariants

  5. 2026-10-14 A self-hosted CI runner has no dependency on the public internet

    now fails loudly A scheduled probe resolves the action registry hostname and reports the DNS error by name; a fallback public resolver removes the single point of failure.

    RuntimeStructuralSynthetic MonitoringObservability Events

  6. 2026-10-14 A local sync server covers hosted download routing

    now fails loudly A bootstrap contract test requires the shard endpoint returned by the sync handshake to reach the download call.

    RuntimeIntegration TestsContract Tests

  7. 2026-10-11 A tool returning output means the command succeeded

    now fails loudly a regression requires exit code 1 to remain a command failure when the tool error flag is false

    RuntimeBehavioralDistributed TracesIntegration Tests

  8. 2026-10-11 A trace span whose status is ERROR records a station failure

    now fails loudly the telemetry relay drops every error span and error log record carrying a tool-call marker, so a routine non-zero exit no longer reaches the specimen queue; a tool call that crashed is dropped by the same rule, which is the hole the last section names

    RuntimeDistributed TracesObservability Events

  9. 2026-10-11 Every exported error occurrence warrants a separate incident task

    now fails loudly regression tests reject historical exports and consumer tool errors, group repeated failures, and enforce a persisted rolling delivery budget

    RuntimeInvariantsObservability EventsRuntime Invariants

  10. 2026-10-11 A queued notification can run with another channel's tools

    now fails loudly a queue regression requires different channel origins to remain queued for their own turns while same-origin follow-ups still fold

    RuntimeInvariantsIntegration TestsRuntime Invariants

  11. 2026-10-11 A deploy that misses its host once is a failed publish

    now fails loudly the deploy retries a dropped connection up to three times and fails fast on any other error

    RuntimeDistributed TracesSynthetic Monitoring

  12. 2026-10-11 A CI runner carries the same tools as a developer's machine

    now fails loudly make test preflights the suite's declared Python dependencies and fails before the demos when one is missing

    RuntimeInvariantsSmoke TestsDependency Graph

  13. 2026-10-10 An Ubuntu host label was safe on a macOS runner

    now fails loudly runner-label-check reads active runner config and pages when a Darwin host runner lacks a matching macOS :host label

    RuntimeInvariantsSynthetic MonitoringRuntime Invariants

  14. 2026-10-10 A restarted worker is assumed to have recovered its queue

    now fails loudly a stalled queue alarms while the worker's message log is still advancing, and a recovery nudge must not report complete while older pending work remains

    RuntimeInvariantsSynthetic MonitoringResource TelemetryRuntime Invariants

  15. 2026-10-10 The token file was assumed to update the running service

    now fails loudly switchboard-watch restarts on /metrics 401, retries authentication, pages if the retry fails, and clears blind-alert state after a successful scrape

    RuntimeInvariantsSynthetic MonitoringRuntime Invariants

  16. 2026-10-10 A busy worker is assumed to be making progress

    now fails loudly a liveness check declares a worker wedged when its message store stops advancing while it is running and busy, then restarts it and pages the operator

    RuntimeInvariantsResource TelemetryRuntime InvariantsSynthetic Monitoring

  17. 2026-10-07 A running worker is assumed to be listening to its queue

    now fails loudly a scheduled liveness check alarms within minutes when a queue's oldest pending item has gone unclaimed by its owning worker past a short limit

    RuntimeInvariantsSynthetic MonitoringResource TelemetryRuntime Invariants

  18. 2026-09-24 A restart policy that fires on exit restarts a hung worker

    now fails loudly a per-harness run timeout now bounds a persistent worker's lifetime, killing and respawning a wedged process and re-arming the timeout every lifetime

    RuntimeInvariantsRuntime InvariantsResource Telemetry

  19. 2026-09-12 YAML frontmatter with a colon-space in a scalar value is a mapping

    now fails loudly Every SKILL.md frontmatter is parsed as YAML by a test; a frontmatter block that does not parse fails CI.

    StructuralRuntimeSchema Validator

  20. 2026-09-06 Looking up a DNS name before creating it is harmless

    now fails loudly Absence is probed only against the authoritative nameserver; the user's own resolver is queried last, and only to read what it already believes.

    RuntimeChangeSynthetic Monitoring

  21. 2026-09-06 Validating a web server's config only reads it

    now fails loudly Config validation runs as the service user, and a smoke request against the new host is what declares the deploy done — not the validator's exit code.

    ChangeRuntimePre-Promotion Invariant GatesSmoke Tests