Valid structured output preserves the source vocabulary
BehavioralStructuralSchema ValidatorIntegration Tests
https://medlab.sublayer.space/c/short-source-coverage/
now fails loudly A sparse normalization fixture must still deliver every short-source label to extraction, and phrase lemmas cannot collapse onto one base word.
What broke
Wortling turns German text and photos into flashcards for language learners. A learner shared a scooter app screenshot containing unfamiliar interface labels. The pipeline transcribed the labels but selected only a subset for vocabulary extraction. It treated app controls as removable boilerplate. A separate phrase candidate represented “go wrong” with the dictionary lemma for “run,” which matched an existing card.
How it showed
The capture finished successfully with one pending card. The stored transcription included Trittbrett, Beschleunigung and Reifendruck. The selected fragments omitted those labels before the known-word filter ran. The learner reported that most of the displayed vocabulary was unfamiliar. The decision log also showed a battery-level compound rejected for rarity and an adjective rejected for English resemblance.
What was watching
Schema validation checked field shapes and candidate limits. Source checks rejected paraphrased sentences and absent target words. Recorded extraction tests covered German morphology with selected sentences already supplied. None required short-source coverage across the normalization boundary. The customer detected the omission during the first successful physical-phone screenshot trial.
Where it sits
This is behavioral data loss despite structurally valid output. The schema validator was present but had no completeness constraint. Integration tests exercised processing stages without an omission fixture. A successful processing status established protocol completion without establishing that all relevant source material reached selection.
What we did
Short captures now retain every nonempty source line for extraction. The code preserves complete model-selected sentences and appends omitted lines verbatim. Long articles retain bounded sentence selection. Phrase lemmas must retain the expression rather than one base word. Versioned prompts stop treating short interface labels as boilerplate and soften frequency and cognate assumptions. A targeted replay can persist an explicit learner request to bypass the model's keep/drop judgment while retaining collection deduplication.
What now fails loudly
A pipeline fixture supplies seventeen scooter labels and a normalizer response retaining only two. Every label must reach extraction without being rewritten. The old implementation fails on the omitted App-Fehler label. A second check rejects a phrase represented as laufen and accepts schief laufen. The old schema fails that check by accepting the collapsed lemma. Both failures were observed by restoring the old implementation before running the new tests.