Caching Strategies

Why cache?

Reading from memory is ~100ns. Reading from disk (database) is ~10ms. That’s a 100,000× difference. Caching is the act of storing frequently-accessed data in fast storage (usually memory) so you don’t hit slow storage every time.

At scale, caching isn’t an optimization — it’s load management. A database that would need 1,000 servers to handle raw query load might handle the same traffic with 10 servers + a cache layer in front.

Cache-aside (lazy loading)

The most common pattern. The application code manages the cache explicitly.

Read:
1. Check cache for key
2. If HIT → return cached value
3. If MISS → query database → store result in cache → return value

Write:
1. Write to database
2. Invalidate (or update) cache entry

async function getUser(userId: string): Promise<User> {
  const cached = await redis.get(`user:${userId}`)
  if (cached) return JSON.parse(cached)

  const user = await db.users.findById(userId)
  await redis.setex(`user:${userId}`, 3600, JSON.stringify(user))  // TTL: 1 hour
  return user
}

Pros: Only caches data that’s actually requested. Cache failures are transparent (fall back to DB). Cache schema can differ from DB schema.

Cons: First request after a cache miss is slow. Cache can become stale if DB is updated without cache invalidation.

Write-through

Every write goes to both cache and database synchronously.

Write:
1. Write to cache
2. Write to database (same operation)
3. Return success

Pros: Cache is always up-to-date. No stale reads immediately after writes.

Cons: Write latency increases (must wait for both cache + DB). Cache fills with data that may never be read.

Use when: Read-heavy workloads where stale data is unacceptable. Pairs well with cache-aside for reads.

Write-back (write-behind)

Write to cache immediately, write to database asynchronously later.

Write:
1. Write to cache → return success immediately
2. Background: flush to database every N seconds (or N operations)

Pros: Lowest write latency. Database writes are batched (efficient).

Cons: If cache fails before flushing, data is lost. Complex to implement correctly. DB and cache can be out of sync.

Use when: High write throughput where some data loss is acceptable (analytics counters, view counts), or where eventual consistency is OK.

Cache eviction policies

Caches have finite memory. When full, something must be evicted.

LRU is the default. It assumes that recently-used data will be used again soon (temporal locality). For most web workloads, this assumption holds.

Redis implements LRU (and LFU) efficiently. Configure maxmemory-policy lru or allkeys-lru.

Cache invalidation — the hard problem

“There are only two hard things in computer science: cache invalidation and naming things.”

The problem: when the underlying data changes, how do you ensure the cache reflects the change?

Option 1: TTL (let it expire) Set a short TTL and accept that cached data may be stale for up to TTL seconds. Simple. Works for data where slight staleness is acceptable (product listings, user profiles).

Option 2: Invalidate on write When data changes, delete the cache key. Next read will miss and repopulate.

async function updateUser(userId: string, data: Partial<User>): Promise<void> {
  await db.users.update(userId, data)
  await redis.del(`user:${userId}`)  // invalidate
}

Option 3: Cache-busting with versioned keys Include a version in the cache key. user:123:v2. Increment version on write. Old keys expire naturally via TTL. Avoids race conditions but uses more memory.

The thundering herd problem: When a cache entry expires, many simultaneous requests all miss and hit the database at once. Solutions: locking (only one request fetches, others wait), probabilistic early expiration (refresh slightly before expiry), or a background refresh process.

Where caches sit in a system

[ Client ]
    ↓
[ CDN ] ← caches static assets (images, JS, CSS) at edge
    ↓
[ Load Balancer ]
    ↓
[ API Server ] ← in-process cache (small, process-local, e.g. LRU map)
    ↓
[ Redis / Memcached ] ← shared cache for all API server instances
    ↓
[ Database ] ← has its own query cache (buffer pool)

Multiple cache layers. Each layer handles different types of data and has different trade-offs.

Redis vs Memcached

Use Redis. Memcached’s one advantage (slightly faster for pure string caching) rarely matters in practice.