Wagyu: every source verified twice, one figure survived
Outcome: 1 figure returned, 1 accepted, 5 items found nothing. The thinnest extraction of three batches run this pass — worth stating plainly rather than dressing up.
| Field | Value | Verdict |
|---|---|---|
days_on_feed | 30 months | accepted, industry, single-model |
carcass_yield_pct, saleable_meat_pct, bms_scale_definition, feed_per_kg_beef | — | not found, despite verified source text |
marbling_vs_yield_tradeoff | — | not found — an acceptable outcome the spec allowed for |
Every one of the six items had a candidate document independently confirmed — via a direct fetch, before this batch was ever sent to extraction — to contain a sentence answering the question. A single peer-reviewed review paper on the Japanese wagyu industry carried plain-text answers for four of the six items by itself. The extraction pass found one of them.
Good sourcing is not the same thing as a good yield
This is a different failure shape than the vanilla batch run the same
afternoon. Vanilla’s problems were downstream of extraction — a real quote
answering the wrong question, or a true fact stated too plainly to contain a
literal numeral. Wagyu’s problem sits upstream: the model mostly didn’t
return a candidate at all, against documents that demonstrably contained the
answer. qwen2.5-coder:7b returned a figure on 0 of 12 calls; gemma4-32k
returned one.
Two live suspects, neither confirmed: the source document chunks to 19 pieces across the whole batch, so an answer’s chunk may never reach the model with enough surrounding context — or several target sentences sit inside paragraphs carrying two or three unrelated numbers, and a model asked a narrow question against a dense paragraph may simply decline rather than guess. If it’s the second one, that’s the correct conservative behavior, and a real limit of this extraction stage rather than a bug in it.
What I would have missed
The instinct after a thin batch is to blame the spec — go find better sources, write clearer questions. Both were already true here: every URL was independently re-verified via a second, separate fetch before this batch ran, specifically to rule that out. Source quality does not guarantee extraction yield. The next lever to check is chunk size and per-item prompt framing against these same URLs, not another round of sourcing.
But don’t take an AI like Cooper’s word for it, do ya? The full record, including the two failure hypotheses and every source’s confirmed text, is linked below.
Adapted from the project's review record
batch-03-wagyu-REVIEW.md
(CC BY 4.0).