medlab / case

medlab

what got hurt, and what now fails loudly

↑ sublayer

2026-11-02 · fixed: yes

A model's quotation of a sentence is the sentence

InvariantsHuman ComprehensionSchema ValidatorProperty-Based Testing

now fails loudly The fidelity check compares words after folding characters that carry no meaning, stores the source's own characters when a sentence matches that way, and reports a validator refusal as a refusal rather than as malformed output.

What broke

A learner photographs or pastes a letter and wortling turns it into cards. The model that normalizes the text is asked to quote the sentence exactly so the card's context is the learner's own material.

A capture that contained a non-breaking space failed three normalization runs. The letter carried one; the model returned the same sentence with an ordinary space. The fidelity check compared characters, refused the capture, and no card was ever made. The same class of character would do it on any other day: a curly quote, an en dash, a ligature, a soft hyphen.

The belief was that a model which reproduces a sentence reproduces its characters. It reproduces its text: the letters and words, rendered with whatever punctuation the model prefers.

How it showed

Three runs, one capture, the same refusal at the normalize step:

Normalize must select verbatim sentences from the source

and nothing in the sentence said which character differed. The learner's letter never became cards, and the retry that the pipeline offers could not help, because the model was answering correctly each time.

What was watching

The fidelity check was the sensor and it fired, which is why the failure was visible at all. It was checking the wrong property: exact characters, where the artifact needs the same words.

The validator around the model's output checked shape, not meaning. Nothing compared the model's sentence to the source in a way that could say "the same words, different punctuation", so the one signal the pipeline had was a refusal with no diagnosis attached.

Where it sits

This is an invariants case with a schema gap.

The family is invariants because the card's property is that its context is the learner's sentence, and character equality is a stronger condition that fails on characters which carry no meaning. It is comprehension because the code assumed the model's rendering and the source's rendering were the same object.

Schema validation was present and checked that the model returned a sentence, not that it was the learner's sentence in the sense the card needs. Property-based testing is the sensor for the relation itself: vary the invisible characters in a fixed sentence and the pipeline's answer must not change.

What we did

The check folds what carries no meaning, using NFKC plus the punctuation a model re-keys, invisible characters, and every run of whitespace, and then compares words. A paraphrase or a re-ordering is still refused.

Where a sentence matches only in that folded form, the source's own characters are stored, so a card quotes the letter rather than the model's rendering of it. A validator rejection also stopped reporting itself as malformed JSON.

What now fails loudly

The pipeline suite carries the capture that failed, with the non-breaking space and the plain one, and requires the card to exist and to carry the source's characters. The golden sentences moved with it, so a change that goes back to comparing raw characters fails a test rather than a learner's morning.

What is not guarded is the set of characters that carry meaning to a German learner and not to a comparison: the fold is a list, and a list is a thing someone maintains. The fold runs on both sides of the comparison and the card stores the source, so a missing character in the list produces a new refusal rather than a corrupted card.