memo 0015: The schema constrains the model. Semantic failure is terminal after one repair.

15.The schema constrains the model. Semantic failure is terminal after one repair.!

The problem

The extractor returns typed items, a todo with a due date, an event with a start, and the rest of the system trusts those types. The interesting question is not how to make the model produce the shape but what happens when what comes back is the right shape and still wrong, and the brief forbids dropping it silently.

The options

A raw prompt plus a hand-rolled parser and repair; a wrapper library that validates with the Pydantic model and re-prompts on failure; forced tool use with a JSON schema, the pattern most tutorials show; native structured outputs, where the API constrains generation to the schema derived from the same Pydantic model that validates the response.

The move

Native structured outputs, one Pydantic model as the single schema for both constraint and validation. Syntax is now a guarantee, so the policy is about semantics. A validation error gets one repair, the same request with the error appended as a new turn. A second failure is terminal. The note moves to failed with the raw output stored, shows in the app as "couldn't process this, retry?", and a retry is a fresh attempt.

What we accepted

Owning the repair loop, which is twenty lines a library would have hidden. Provider lock-in at the schema boundary, since the constraint parameter is one vendor's. Two model calls before a semantic failure surfaces, which is why the repair count is one and not three. A column that exists only for the bad path, which is the difference between "it failed" and knowing why without re-running.

When we would revisit

When repairs succeed more than about a fifth of the time on the golden set, which would mean the first prompt is underspecified and the fix belongs in the prompt, not the loop.

A validation failure is classified as terminal after repair rather than retryable because it is deterministic. The same prompt on the same text produces the same shape again, and retrying it with backoff only burns money on the way to the same failed state. The split that matters is syntactic against semantic: the API can guarantee the first, and the second is ours.

One finding from the eval set shaped the schema itself. With constrained decoding, every field in the output schema is required, defaults included, because an optional field declared second let the model skip it, write the next one, and smuggle the confidence into the summary string. The JSON was valid. Only a semantic check caught it.