CI gate for GitHub Actions
A first-party action fails the build when a change scores below your threshold. Two lines of YAML.
What changed
lattis/evals-action, a first-party GitHub Action that runs your eval suite on every pull request.
Pull request checks now show the suite score, the threshold and the three lowest-scoring cases inline.
Add the action to your workflow and set a threshold. Suites without a threshold report scores but never fail the build.
Release notes
The new lattis/evals-action runs your eval suite on every pull request and fails the check when a change scores below your threshold.
The action wraps the Lattis CLI, so the same check runs in GitLab CI, CircleCI, Buildkite or any runner that can call a command.
Questions about this release
The docs cover every flag and endpoint. For anything else, support answers within one business day.
Run your first agent in five minutes.
1M free tokens a month. No card, no sales call, production keys in two minutes.