Structured output & extraction
Getting well-formed data out of a model is a solved problem. There's a way to make malformed output structurally impossible, and if you're still maintaining a parser with fallbacks, you're solving something that stopped being a problem a while ago. What that doesn't solve is the invoice whose line items don't add up to its own total, or the field filled in confidently with something the document never said. That second layer is where extraction systems are actually won or lost.
You probably need this if
- Your parser has a try/catch around it and a retry that mostly works.
- The format drifts between runs no matter how detailed the instructions get.
- Required fields come back populated with plausible fiction.
- Reviewers dismiss almost everything from one category, and now distrust the rest too.
- A nightly pass over everything would be genuinely useful and costs too much to run.
What you get
- Malformed output, designed out
- Structure the request so the model can't hand back something unparseable in the first place, and an entire category of failure disappears. Everything downstream gets simpler the same day.
- Schemas that don't invite invention
- Mark a field required and the model will fill it, even when the document says nothing — that isn't lying, it's following instructions. Fields that might legitimately be absent are allowed to be absent, genuine ambiguity gets a value that means "unclear," and categories that will grow get room to grow.
- Examples where they earn their keep
- Two or three worked cases covering the genuinely ambiguous inputs, showing the reasoning and not just the answer. That generalizes to documents you haven't seen yet. More instructions don't.
- The checks a schema can't make
- A perfectly well-formed answer can still be wrong. We have the model surface what it calculated next to what the document claimed, so mismatches announce themselves — and when something does need another pass, it goes back with the specific complaint attached rather than a vague request to try harder.
- Volume that pencils out
- For work that can wait, there's a path at roughly half the price. Sized so the turnaround still clears your deadline, and built so one bad document costs you that document rather than the whole run.
The judgment behind it
These are the calls that decide whether a system survives contact with production or just demos well. They are not obvious, and getting them wrong is rarely obvious either — until it is.
- "Be careful" is not an instruction.
- Telling a model to be conservative, or to report only what it's confident about, does close to nothing. Telling it precisely what counts and what doesn't works. Rules beat encouragement, every time.
- Retrying can't conjure what isn't there.
- If the number lives in a document nobody attached, no number of attempts will produce it. Knowing which failures are worth retrying is the whole difference between a pipeline that self-corrects and one that's expensively stuck.
- One noisy category poisons the rest.
- People don't quietly filter by category. They conclude the tool is unreliable and stop reading all of it. Switching off the noisy check protects the ones that work while you go fix it.
- A single accuracy number hides the problem.
- Ninety-seven percent overall is entirely compatible with near-total failure on one document type nobody thought to check separately. Accuracy only means something broken out.
Engagement shape
Usually 3–6 weeks: schema and prompt design, a test harness with a labelled set so you can see accuracy move, a batch path where the volume justifies it, and results reported by segment rather than as one number.