A scaling playbook in order: measure, cache reads, add replicas, queue writes, partition last

Scaling advice often comes as a list of techniques with no order: add a cache, use replicas, shard the database, go event-driven. Every one of those is useful, and each has a cost that is easy to underestimate. The order in which you apply them matters more than any single technique.

This is the order we follow for a typical web service backed by a relational database, refined over a few years of growing a marketplace through several sharp traffic increases. For each step: the signal that tells you it is needed, what to do, and what it costs.

Step 0: measure where the time goes

Signal: something is slow or about to be, and you do not yet know why.

Before changing architecture, find the actual bottleneck. Our standard first pass:

• Request breakdown: distributed traces or APM showing how much of each slow endpoint's time is spent in the database, in other services, in application code and in waiting for resources.
• Database view: top statements by total time (pg_stat_statements or MySQL's performance_schema.events_statements_summary_by_digest). Total time, not mean time: a 5 ms query called a million times costs more than a 2 s query called once.
• Saturation: CPU, memory, connection pool usage, queue depths, disk I/O. Which resource is close to its limit?

Most of our "we need to scale" moments ended at this step. A missing index, an N+1 query in a list endpoint, a pool that was too small, a log statement in a hot loop. Fixing those costs hours, not weeks, and often buys more than any architectural change would.

Cost: low. The real risk is skipping it.

Step 1: scale the stateless tier

Signal: application servers are CPU-bound or near their request concurrency limit, and the database is not the bottleneck.

If your web servers keep no state in memory between requests (sessions in a shared store, uploads in object storage, no local caches that must agree), adding more instances behind the load balancer is the easiest scaling there is. Autoscaling on CPU or request rate handles daily peaks.

Cost: low, if the tier really is stateless. If it is not, this step forces you to fix that first: sticky sessions, local file writes and in-memory caches all break when there are many instances. More app servers also mean more database connections, so watch the database's connection limit; this is where a connection pooler like PgBouncer often becomes necessary.

Step 2: cache reads

Signal: the database is busy, and most of its load is reads of data that changes far less often than it is read.

For us, product pages are read thousands of times for every update. Caching the rendered product data in Redis took most product reads off the database.

What we cache, in order of preference:

1. Immutable or versioned data (static assets, fingerprinted files, historical records): cache forever, no invalidation needed.
2. Data that tolerates short staleness (product listings, counts, recommendations): cache with a short TTL and accept the staleness.
3. Data that must be fresh after writes (prices, stock, permissions): cache with explicit invalidation, which is where bugs live. Do this last and test the race conditions.

Cost: moderate. Every cache is a second copy of the data that can be wrong. It needs invalidation logic, protection from stampedes when popular keys expire, and monitoring of hit rate. A cache with a low hit rate adds latency and complexity for nothing.

Step 3: read replicas

Signal: read load is still too high after caching, typically from queries that are hard to cache (search, filtering with many combinations, reports, admin tools).

Replicas copy the primary's data asynchronously and serve reads. We route reporting, exports, search filters and admin list pages to replicas.

Cost: moderate, mostly in correctness. Asynchronous replicas lag behind the primary, usually by milliseconds and occasionally by seconds or more. Any read that must see the user's own recent write (read your own writes) must go to the primary, or wait until the replica has caught up. We route by default to the primary for a few seconds after a user writes, then back to replicas. Replicas also do not help with write load at all; every replica applies every write.

Step 4: move writes off the request path

Signal: the primary is busy with writes, or requests are slow because they do work the user does not need to wait for.

Not every write must finish before the response. Sending emails, updating search indexes, recalculating recommendations, writing audit events, calling webhooks: all of these can go into a queue and be done by workers. The request writes the essential data, commits, enqueues the rest, and responds.

Queues also smooth peaks. A flash sale that would produce 5,000 writes per second for one minute can be absorbed by a queue and processed at 1,000 per second over five minutes, if the business can accept the delay for those writes.

Cost: high in complexity. You now have eventual consistency to explain to users ("your export is being prepared"), at-least-once delivery to handle (idempotent consumers), poison messages, dead-letter queues, and monitoring of queue depth and age. For writes that must be consistent with the main transaction, you need an outbox or similar pattern so the database commit and the enqueue cannot disagree.

Step 5: scale the database vertically

Signal: the primary is the bottleneck, and the steps above have taken what they can.

We list this here rather than earlier because it is easy to reach for too soon, but it deserves to come before partitioning. A larger database instance, faster storage, more memory for the working set, is a configuration change. Modern hardware goes a long way: a single well-tuned PostgreSQL or MySQL server handles tens of thousands of transactions per second for typical OLTP workloads.

Cost: money, and a ceiling. Each size up costs more, and eventually there is no bigger instance. But compared with partitioning, it buys time with almost no engineering effort.

Step 6: partition the data

Signal: the write volume or data size exceeds what one primary can handle, even the largest one you are willing to pay for, and you have exhausted the steps above.

Partitioning (sharding) splits the data across several independent databases, each owning a subset of keys, typically by tenant or by hashed ID.

Cost: very high, and mostly permanent:

• Queries across partitions become scatter-gather or impossible.
• Transactions across partitions need distributed coordination or careful redesign.
• Choosing the partition key wrongly creates hot partitions, and changing it later is a migration project.
• Operations (backups, schema changes, failovers) multiply by the number of partitions.

Before sharding a whole database, look for a smaller split. Often one table (events, logs, messages) accounts for most of the write volume. Moving just that table to a store designed for it, or partitioning just that table, solves the problem without touching the rest.

Signals at a glance

Mistakes we made by skipping ahead

• Sharding before measuring. An earlier team split the orders database by region. Most of the load turned out to be from three report queries that a replica could have served. The regional split stayed, along with every cross-region report that now needed custom merging code.
• Caching before fixing queries. We cached a slow product search result. Cache misses were still slow, and with many filter combinations, most requests were misses. Adding the right index made the uncached query fast enough that the cache could be removed.
• Replicas without lag handling. Users saved settings and immediately saw old values, because the next page read from a replica. It looked like the save had failed, and people saved twice.
• Queues without dead-letter handling. One malformed message blocked a queue for an afternoon.

How we decide

Every proposal to scale something now comes with three items: the measurement that shows the bottleneck, the step from this list it corresponds to, and why the earlier steps are not enough. That conversation is short when the data is there, and it has stopped us from building infrastructure we did not need more than once.

Summary

• Measure first. Most scaling problems are a query, an index or a configuration away from solved.
• Scale stateless app servers horizontally; it is cheap if they really are stateless.
• Cache reads, then add replicas for reads that cannot be cached, and handle replication lag.
• Move non-essential writes to queues, and make consumers idempotent.
• Buy a bigger database before partitioning. Partition last, and as narrowly as possible.

References: PostgreSQL pg_stat_statements, PostgreSQL high availability and replication, MySQL replication.