Saffron: the first batch through the pipeline

2026-07-30

The first batch to go all the way from a machine extraction run to live corpus. Outcome: 5 figures returned; 3 accepted as-is, 1 reframed, 1 dropped, 1 added by a human.

The dropped row is the one that matters

The model answered flowers_per_gram_dried with a quote about stigmas per pound. The quote was real, verbatim, and in the right document — it simply answered a different question. A crocus has three stigmas per flower, so the error was wrong by exactly the factor this subject exists to measure.

Verbatim-quote verification cannot catch this. It proves a sentence exists; it says nothing about whether that sentence answers the question asked. That is what the pipeline’s unit-matches-field check was added for, and this row is the reason it exists.

A finding reports a source; a corpus row models it

drying_mass_yield: 0.2 was rejected because 0.2 appears nowhere in its quote (“the stigmas lose 80% of their weight”). The arithmetic is right; the reporting was not. The fix was not to relax the gate. It was to record what the source says — a drying mass loss of 0.8, which the value check accepts against “80%” — and put the inversion in the loss-chain data, where a human wrote the survival rate with a comment explaining it.

It was also graded industry, not derived: inverting a trade rule of thumb yields a trade rule of thumb. In this corpus derived means computed from measured figures, and this source measured nothing.

Retrieval worked; extraction did not

The correct flowers-per-gram figure — 150, stated outright — was sitting in a document the pipeline had already fetched and never extracted from. A human found it by reading the returned artifact. The fetch and embedding stages found the right source; the extraction stage read only one PDF and ignored the other two documents. This is the concrete justification for returning raw documents with every batch: without the artifact, the figure would have been unrecoverable and the misattributed row would have looked like the best available answer.

A conflict recorded rather than resolved

Two land-grant extension services disagree about per-acre yield roughly three-fold — Penn State’s “about 3 pounds” per acre versus UF/IFAS’s “8–10 pounds … in an established planting”. They are probably not measuring the same thing: a saffron field takes years to establish. Neither is loaded as data. The disagreement itself is recorded, because a three-fold spread between two extension services is the most honest thing we currently know about saffron yields.

The cross-check that makes the subject trustworthy

Two sources, two different routes, 3% apart: UC ANR states 150 flowers per gram; the UF/IFAS figures imply 154.3 (210,000 stigmas ÷ 3 per flower ÷ 453.59 g/lb). Different publications, different decades, different framings — and the agreement is pinned by a test.

What going first cost

Translating four verified figures into live corpus took roughly the same effort as the extraction run that produced them — and it surfaced a real model bug the poultry corpus could never have exposed: the pooling maths assumed every contributing individual gives at least one whole unit. True of wings and eggs; false of a gram of saffron, where one run reported “came from about 1 different flower” against a floor of 150. So “a new subject is four YAML files and zero Python” was wrong: it was four YAML files, one model fix, one CLI fix, and 16 tests. Cheaper to learn on the second domain than the fifth.

Adapted from the project’s review record batch-01-saffron-REVIEW.md (CC BY 4.0).