developer-roadmap/roadmaps/inference-engineering/content/scale-to-zero@Pi87dWgu5NTIJRQUc169s.md

Scale to Zero

Scale to zero allows the system to run zero replicas during periods of no traffic and spin up on demand. It requires fast cold starts and robust queuing to hold incoming requests while a replica starts. Scale to zero is well-suited for development environments and periodically accessed workloads, but not for latency-sensitive production applications with unpredictable light traffic.

Visit the following resources to learn more:

• @article@Should LLM Serving Scale to Zero for Cost
• @article@Runpod Secrets: Scaling LLM Inference to Zero Cost During Downtime
• @article@Zero Redundancy Optimizer (ZeRO): Scaling LLM Training Without Breaking the Bank