The challenge
Veda Health summarises patient intake notes for clinicians. Their safety board would not approve a new model without proof that it was at least as good as the old one on real cases.
The change
The team built a versioned test set of 2,400 cases from redacted production traffic, graded by clinical rubrics. Every prompt or model change now runs through the Lattis CI gate, with inference pinned to the EU region.
evals:
suite: clinical-intake-v4
cases: 2400
region: eu-west
gate: score >= baseline
The review board wanted numbers. Evals gave us numbers.
Dr. Priya Raman
,
Clinical AI Lead
,
Veda Health
The result
The review board approved the switch with the eval report as evidence. Since then, no change that lowered the score has reached production.
