· Architecture · 5 min read
Scale-to-zero without surprising user-facing traffic
By Ethan Cole, Co-Founder & CTO
Autoscaling is only helpful when it respects clear floors and ceilings. Here is how we separate background workers that can sleep from APIs that cannot.
Scale-to-zero sounds universally attractive until an interactive checkout endpoint stalls for four seconds waking up a container before breakfast.
In practice, workloads fall into two very different buckets. Preview environments, cron runners, and asynchronous queue workers handle cold starts gracefully and should drop to zero replicas whenever idle.
Customer-facing web servers and core APIs rarely belong at zero. Leaving a floor of one replica removes cold-start latency for real users while still letting Nimbus scale out on CPU, memory, or request rate when traffic climbs.
Just as important is the ceiling. Every autoscaling rule in Nimbus asks for both a minimum and a maximum replica count, so a runaway client retry loop cannot multiply instances indefinitely overnight.
Good scaling policy is mostly about setting honest bounds per service. Let idle workers sleep, keep interactive paths warm, and cap the top end before a spike turns into a billing surprise.