medlab / case

medlab

what got hurt, and what now fails loudly

2026-09-12 · fixed: partial

An AI reviewer that reports an approval has posted one: a review created without a submit event sits as an invisible draft, and the record says nobody reviewed anything

ChangeHuman ComprehensionIndependent ReviewSecond-Agent Review

now fails loudly A review counts only when the forge API shows it submitted with a verdict; until the tool submits, the merge record is the evidence and the reviewer's own result is not.

What broke

The setting, in the words a stranger needs. One Linux box runs a self-hosted git forge — Gitea, a GitHub-like server for repositories and pull requests. On the same box, two long-lived AI coding agents run as persistent processes under a small supervisor that keeps them alive. One, the worker, takes tasks and opens pull requests. The other, the reviewer, has one job: read each pull request, run the tests, and post a verdict. A small queue turns forge events — a pull request opened, an issue filed — into tasks for them.

The agents do not call Gitea's HTTP API by hand. They go through a thin command-line tool written for them, forge, which wraps that API with commands for listing, creating and merging pull requests and reading CI. The reviewer's standing instructions say: read the diff, run the tests, post the review with that tool, never hand-write raw API calls.

It did all of that all afternoon, and reported an approval on every PR it finished.

Gitea's API, asked for the reviews on those pull requests, returned an empty list for eight of them and a single review in state PENDING for the ninth.

How it showed

Nothing showed. That is the case. The reviewer's task results said verdict: approve. The PRs merged. The timeline on one PR carried a review event with the reviewer's text in it. Only a direct read of GET /repos/{owner}/{repo}/pulls/{n}/reviews — done while building the activity feed for the setup's public status page, which wanted to say who had reviewed what — showed that the approval had been created and never submitted.

Gitea's review model explains the state. A review created with a body and no event — no APPROVE, REQUEST_CHANGES or COMMENT — is staged, not posted: the forge holds it as a draft under the account that created it, invisible to anyone else, and it never becomes an approval until something submits it. At the time both agents acted through the owner's single Gitea account, so the draft sat there under that name, seen by no one.

The tool the instructions name has no review command at all. Its command list is pr list | show | create | merge, checks, wait, whoami. Whatever path the reviewer found to leave a draft, it was not the tool's, and the instructions had no way to notice.

Where it sits

Every case here is mapped onto the Software Observatory, a catalog — not ours — of "sensors": kinds of checks that can each tell you something true about a piece of software, grouped into families of failure. A case names the families it falls in and the sensors that would have seen it.

A change control that was believed to be in force and was not, and a comprehension failure: the record of what was reviewed did not match what happened, and no one reading the record could tell. The observatory's independent review sensor is the plain thing: someone other than the author reads a change before it lands. It is the sensor this setup thought it had running. Its second-agent review sensor is the specific form in play here: a second AI agent reads the first agent's change. Both were operating in the sense that reading happened and a verdict was formed. Neither was operating in the sense that matters, which is that the verdict is recorded somewhere the next decision can see it. Gitea's branch protection, with a required-approvals rule, would have reported these PRs as unreviewed and refused to merge them. So would any audit.

What we did

Gave the reviewer its own Gitea account, so a review it does submit is recorded under its own name and it can no longer review its own work by accident. Changed its instructions so that an approval merges: the reviewer's own merge is now on the record as merged_by, which is the one durable trace of the review having happened. Filed the missing command against the tool: a pr review command that posts with a verdict event set, and refuses when the token's user is the PR author.

The command is not built yet, which is why this case is marked partly fixed. Until it lands, the merge is the evidence, and the activity feed that started this investigation shows reviews as empty rather than pretending.

What now fails loudly

  • A verdict that is not on the record is not a verdict. The reviewer's

result now has to cite the review's own URL from the API, not its intention. A result without one is a failed task, not a completed one.

  • The feed reads the API, never the reviewer's report. An approval

appears on the public status page's activity feed only when Gitea returns it as submitted. The feed showing nothing is the alarm.

The false belief, in one line: the tool I was told to use has the command my instructions assume. The evidence that contradicted it was one --help away and nobody had run it.