Every model call in this pipeline goes through one function.

Not a loose convention. One choke point: prompt generation, parsing, adjudication, recommendations, all wired through one wrapper around LiteLLM. No service touches a provider SDK directly.

That choke point exists for the obvious reason, swapping models without touching business logic. Looking closely at what sits on top of it revealed something unexpected.

The guarantee that wasn't load-bearing

Structured output looked like it was doing real work in the pipeline. A response format, a schema, something the provider enforces before the answer comes back. The pipeline never uses one.

Every JSON extraction step asks for JSON in the prompt and parses whatever it gets. When parsing fails, it falls back to a template or keeps the raw text instead. Nothing crashes. Nothing is guaranteed either.

That looks like a weak spot until you see what catches the errors. A separate deterministic pass checks every model verdict against the source text it came from, independent of whether the JSON parsed cleanly. The cross-check is the safety net. Schema compliance never was.

Tool calling tells the same story from the other side. Zero tool_choice in the codebase. Zero function_call. No normalization layer for provider tool formats exists, because nothing in the pipeline ever calls a tool.

Error handling, the one conventional part

One classifier, one place, retryable versus non-retryable, based on status codes and a short list of exception names for providers that don't report consistently. It doesn't split content-filter errors from context-length errors or auth errors. All three fail the same way. No crash. A clean failed result, nothing pretending to succeed.

None of this was a plan from day one. It fell out of building the pipeline for a different reason. What holds it together only became clear after checking it against the code instead of what seemed likely to be there.

P.S. Anything built on a provider guarantee is worth this question: check what happens the day that guarantee quietly stops holding. Here, the guarantee that seemed load-bearing turned out not to be part of the safety mechanism at all.