Design an Async Request Cache

A practice problem in the escalating-screen format — and the async axis most prep sets miss entirely. For any team that owns a data layer, this is one of the most plausible things to be asked, and it's still a data-structures problem: a Map, a state machine, and eviction. It just wears a Promise.

Phase 1 — the prompt

A frontend hammers the same endpoints from many components at once. Build the layer that stops that from becoming many identical network calls.

type Fetcher<T> = () => Promise<T>;

class RequestCache {
  get<T>(key: string, fetcher: Fetcher<T>): Promise<T>;
}

Rules:

• First call for a key runs the fetcher.
• Calls for the same key while the first is still in flight wait on that same in-flight work. The fetcher runs once; every caller resolves with the same value. This is the whole point. Handing back the stored promise or a promise derived from it (stored.then((v) => v)) both satisfy it — what must not happen is a second fetch.
• Once resolved, the value is cached: later calls return it without running the fetcher again.
• Different keys are independent.

Before writing, answer out loud: what exactly is stored in the map? The naive answer is "the value." But think about what you actually have at the moment the second caller arrives — the fetch hasn't finished. There is no value.

Phase 1 — the debrief

What you have in-flight is the promise, so that's what you store:

class RequestCache {
  private entries = new Map<string, Promise<unknown>>();

  get<T>(key: string, fetcher: Fetcher<T>): Promise<T> {
    const existing = this.entries.get(key);
    if (existing) return existing as Promise<T>;

    const promise = fetcher();
    this.entries.set(key, promise);
    return promise;
  }
}

Storing promises collapses two states — "in flight" and "resolved" — into one representation, and dedup falls out for free: the second caller gets the same promise object, so the fetcher can't run twice. A resolved promise hands its value to every later .then() immediately, so the "cached value" case needs no separate code path.

The subtle discipline: set the map synchronously, before any await. An async implementation that awaits first opens a window where two callers both miss the map and both fetch — the exact bug the layer exists to prevent.

Escalation 1 — the fetcher rejects

Right now a rejection is either cached forever or crashes something. Decide the semantics and defend them: a rejected request must not be cached; every caller waiting on the in-flight request sees the rejection (nobody hangs); after a failed attempt, the next call retries.

  get<T>(key: string, fetcher: Fetcher<T>): Promise<T> {
    const existing = this.entries.get(key);
    if (existing) return existing as Promise<T>;

    const promise = fetcher();
    this.entries.set(key, promise);
    promise.catch(() => {
      // only evict if this promise is still the tenant — a retry may have replaced it
      if (this.entries.get(key) === promise) this.entries.delete(key);
    });
    return promise;
  }

Everyone already holding the promise sees the rejection (correct — their request genuinely failed); the delete means the next caller retries. The identity check before deleting protects a fast retry from being evicted by the stale failure's cleanup.

The interviewer's real question is "why?" Have the sentence ready: a cached success is a fact; a cached failure is a guess about the future. Then name the exception before they do — a 404 is a fact about the resource and cacheable; a timeout is a fact about the network a moment ago and isn't. If you cache negative results, cache them by kind, briefly.

Escalation 2 — entries go stale

Add a TTL: get(key, fetcher, ttlMs?). Older than TTL → refetch. No TTL → never expires. Expiry must not depend on a timer firing — check on read. An expired entry being refetched must still deduplicate.

Store { promise, expiresAt }; on read, treat an expired entry as a miss and replace it — the replacement promise is what concurrent callers then share, so dedup survives expiry. Timestamp the entry when the promise resolves, not when the fetch starts, or slow fetches are born half-expired.

Why check-on-read is the right call — say it unprompted: a timer per entry is a scheduling problem and a leak (the timer pins the entry), and nobody is harmed by a stale entry nobody reads. Lazy expiry is how Redis does it, for the same reason.

Escalation 3 — bound the memory

The cache is unbounded in a long-lived process. Add a capacity; evict least-recently-used. A get counts as a use, including a hit. An in-flight request must never be evicted — its callers are still waiting on it. Capacity of one must work.

This is LRU Cache wearing a different hat — and in TypeScript there's a shortcut worth saying out loud: Map preserves insertion order, so delete-then-reinsert moves a key to the most-recent end, and the first key the iterator yields is the LRU:

  private touch(key: string, entry: Entry): void {
    this.entries.delete(key);       // Map preserves insertion order:
    this.entries.set(key, entry);   // delete + reinsert = move to most-recent
  }

  private evictIfNeeded(): void {
    while (this.entries.size > this.capacity) {
      for (const [key, entry] of this.entries) {      // iterates oldest-first
        if (!entry.inFlight) { this.entries.delete(key); break; }
        // in-flight entries are skipped: their callers are still waiting
      }
    }
  }

The in-flight guard is the escalation's real content: evicting a pending entry doesn't cancel the fetch (the promise is already out there) — it just forgets the dedup, so the next caller starts a second identical request while the first is still running. The bug the whole class exists to prevent, reintroduced by the eviction policy. (Pedantic edge to name: if everything is in-flight at capacity, you briefly exceed capacity rather than break dedup — state the choice.)

Escalation 4 — pick your ending

Cancellation. get takes an AbortSignal. One caller aborting must not cancel the shared request for everyone else — only when every waiter has aborted does the underlying fetch die. You're refcounting waiters per entry; the leak to name is the waiter who neither aborts nor settles.

Server-side rendering. The same cache runs during SSR, where the process is shared across all users. Name the security problem before any code: a module-level cache keyed user:me serves one user's data to the next request. That observation is worth more than any implementation — it's the failure mode data-layer teams actually live with. The fix: per-request cache instances (a new cache per incoming request), and what it costs you is cross-request warm hits, which is exactly the price of correctness here.

What's being scored

• "Store the promise, not the value" — the one insight phase 1 exists to find.
• Setting the map synchronously; no await-shaped race window.
• Failure semantics defended with the fact-versus-guess sentence.
• Recognizing escalation 3 as LRU, and the Map insertion-order idiom.
• SSR cross-user leakage named as a security bug, unprompted.