The challenge
Northwind ran every request through one provider, at one price, with no fallback. Two outages in a quarter took their assistant offline, and the inference bill grew faster than revenue.
The change
The team pointed their client at Lattis and set a quality threshold of 0.92, measured on 1,800 real support conversations. Routing now picks the cheapest model that clears the bar, request by request.
router:
model: auto
quality: 0.92
fallback: [open-70b, frontier-xl]
eval_suite: support-v3
We expected a migration. It was a config change.
Maya Lindqvist
,
CTO
,
Northwind Labs
The result
Costs fell 62% in the first two weeks. Median latency settled at 38 ms, and two provider outages since then were absorbed by fallback with no customer impact.
