Scaling Fundamentals
One server handles your traffic until it doesn't. What you do next, and in what order, is most of what "scaling" means.
Prerequisites: A web server, a database, and one HTTP request in your head.
After this: Explain when to scale up versus out, and what has to become stateless first.
Suggested first pass: Read sections 1–5, answer each section in your own words, then use the remaining failure modes and exercises as the advanced pass.
Start with one server running your app and one database beside it. It works. Traffic grows. Something runs out, and there are only four candidates:
- CPU: too much work per request, or too many requests.
- Memory: the working set no longer fits, so the machine starts swapping or the cache stops helping.
- Disk: out of space, or out of IOPS.
- Network: the box cannot push bytes fast enough.
Knowing which one is saturated matters more than any diagram. Adding servers does nothing for a system that is slow because one query does a full table scan. Most "we need to scale" problems are missing-index problems.
If CPU sits at 20% while requests are slow, the bottleneck is waiting: on the database, on a downstream service, on a lock. More app servers will not fix waiting. They give you more things that wait, and more connections into the database that is already the constraint.