medlab / case

medlab

what got hurt, and what now fails loudly

↑ sublayer

2026-09-12 · fixed: partial

An AI reviewer that reports an approval has posted one

ChangeHuman ComprehensionIndependent ReviewSecond-Agent Review

now fails loudly A review counts only when the forge API shows it submitted with a verdict; until the tool submits, the merge record is the evidence and the reviewer's own result is not.

What broke

One Linux box runs a self-hosted git forge, Gitea. On the same box, two long-lived AI coding agents run as persistent processes under a small supervisor. One, the worker, takes tasks and opens pull requests. The other, the reviewer, has one job: read each pull request, run the tests, and post a verdict. A small queue turns forge events — a pull request opened, an issue filed — into tasks for them.

The agents do not call Gitea's HTTP API by hand. They go through a command-line tool written for them, forge, which wraps that API with commands for listing, creating and merging pull requests and reading CI. The reviewer's standing instructions said: read the diff, run the tests, post the review with that tool, never hand-write raw API calls. The tool's actual command list is pr list | show | create | merge, checks, wait, whoami — it has no command for posting a review at all, a gap the instructions did not account for and the reviewer's own output gave no sign of.

The reviewer read pull requests and reported an approval on each one it finished for an afternoon. Queried directly, Gitea's review API returned an empty list of reviews for eight of those pull requests, and a single review in state PENDING for the ninth.

How it showed

The reviewer's task results said verdict: approve. The pull requests merged. The timeline on one of them carried a review event with the reviewer's own text in it. Nothing in any of that distinguished a real review from what had actually happened, until a direct read of GET /repos/{owner}/{repo}/pulls/{n}/reviews — done while building an activity feed for the station's public status page, which wanted to name who had reviewed what — returned reviews that had been created and never submitted.

Gitea's review model explains the state. A review created with a body and no event — no APPROVE, REQUEST_CHANGES or COMMENT — is staged, not posted: the forge holds it as a draft under the account that created it, invisible to anyone else, until something submits it. Both agents acted through the same single Gitea account at the time, so the draft sat there under that shared name, visible to neither of them as a problem.

What was watching

The independent review and second-agent review sensors were both on the path and both silent. The reviewer read each pull request, formed a verdict, and reported an approval — the reading and the verdict-forming both happened. Neither sensor caught the gap that mattered: the verdict was never recorded anywhere the next decision (a merge, branch protection, an audit) could see it.

A human caught the failure while building the activity feed described above, a day after the affected pull requests had merged.

Where it sits

This is a change control believed to be in force that was not, and a comprehension failure: the record of what was reviewed did not match what had happened, and nothing reading the record could tell the difference. Gitea's branch protection, with a required-approvals rule, would have reported these pull requests as unreviewed and refused to merge them; so would any audit that checked the review API instead of the reviewer's own report.

What we did

The reviewer now has its own Gitea account, so a review it does submit is recorded under its own name and it cannot review its own work by accident. Its instructions were changed so that an approval merges the pull request itself: the reviewer's own merge is now on the record as merged_by, the one durable trace that a review happened. A missing command was filed against forge: pr review, which posts with a verdict event set and refuses when the token's user is the pull request's author.

That command is not built yet; this case is marked partly fixed. Until it lands, the merge record is the evidence, and the activity feed that surfaced this shows reviews as empty rather than reporting the reviewer's unsubmitted intent.

What now fails loudly

  • A verdict that is not on the record is not a verdict. The reviewer's

task result now has to cite the review's own URL from the API, not its intention. A result without one is a failed task, not a completed one.

  • The activity feed reads the API, never the reviewer's report. An

approval appears on the public status page only when Gitea returns it as submitted, so a reviewer that stops submitting shows up as a reviewer that stopped reviewing, not as a page full of approvals nobody can see.