Running 1.2M sandboxes a month:lessons from Helio.

Helio runs a coding agent for 40,000 developers. Every session gets its own sandbox, which adds up to 1.2 million sandbox runs a month. Here is what we learned getting there.

Cold starts

Our first version booted a fresh image for every run and spent four seconds before the agent did anything. Pre-warmed pools brought cold starts under a second, and developers stopped noticing them.

Network policy

We started with open egress and closed it one incident later. Each workspace now has an allow list, and agents can only reach the package registries and APIs on it:

network:
  allow:
    - registry.npmjs.org
    - pypi.org
    - api.helio.internal

Failure mode one: runaway loops

An agent that retries a failing test forever burns minutes quietly. A per-run spend limit stops it, and the trace shows exactly where it looped.

Failure mode two: slow tools

One slow internal API held hundreds of sandboxes open. Timeouts per tool, not per run, fixed it. Our checklist today:

• Pre-warmed pools sized to the busiest hour
• An egress allow list per workspace
• A spend limit on every run
• A timeout on every tool

“Isolation we could explain to security was the thing we could not build ourselves.”