Scaling Fundamentals
One server handles your traffic until it doesn't. What you do next, and in what order, is most of what "scaling" means.
Prerequisites: A web server, a database, and one HTTP request in your head.
After this: Explain when to scale up versus out, and what has to become stateless first.
Suggested first pass: Read sections 1–5, answer each section in your own words, then use the remaining failure modes and exercises as the advanced pass.
Content reviewed 21 June 2026 · References: AWS Well-Architected Framework: Reliability pillar · Google SRE: Addressing cascading failures
Start with one server running your app and one database beside it. It works. Traffic grows. Something runs out, and there are only four candidates:
- CPU: too much work per request, or too many requests.
- Memory: the working set no longer fits, so the machine starts swapping or the cache stops helping.
- Disk: out of space, or out of IOPS.
- Network: the box cannot push bytes fast enough.
Knowing which one is saturated matters more than any diagram. Adding servers does nothing for a system that is slow because one query does a full table scan. Most "we need to scale" problems are missing-index problems.
If CPU sits at 20% while requests are slow, the bottleneck is waiting: on the database, on a downstream service, on a lock. More app servers will not fix waiting. They give you more things that wait, and more connections into the database that is already the constraint.