Four timeline smells

A dense GPU lane means the kernels can be the story. White space means they are not. Classify first; do not rewrite attention in the blank regions.

The four traces (ASCII is enough)

Trace 1 — picket fence

GPU: k k k k k k k k k k
      ^gaps of 8–20 µs between 3 µs kernels, repeating 128 times (decode)
CPU: python loop visible between launches

Trace 2 — host island

GPU:  [==== prefill 42 ms ====]            [==== decode step ====]
CPU:                         [-- tokenize 28 ms --][-- json 9 ms --]

Trace 3 — copy on the path

GPU:      [H2D 6 ms][kernel 4 ms][D2H 5 ms][kernel 4 ms]
CPU:  np.array ← tensor.cpu()  then later  tensor(batch)

Trace 4 — sync cliff

GPU: [matmul][attn]          [matmul][attn]
CPU:               .item()  wait  .item()  wait

Part A — Cards

For each trace write:

Smell:
Cause class: launch | host | copy | sync
Why the GPU is empty:
First experiment:
Forbidden next step:

Part B — Decode product

Which two traces are the default decode failure modes at batch 1? What serving change attacks both at once?

Part C — Protocol

Copy the six-step protocol from the lesson from memory. Apply it to Trace 3 as if this were a Monday incident.

Acceptance

1. Trace 1: graphs/compile/batch; not a new MMA kernel.
2. Trace 2: move tokenize/post off the request critical path or overlap; not cuBLAS.
3. Trace 3: keep tensors on device; .cpu() / numpy is the bug.
4. Trace 4: .item() / print / implicit sync; CUDA events instead.
5. Continuous batching + compiled/graph decode is the "both" answer for 1 and 4.

Stretch

Add NVTX range names you would inject for a prefill+128 decode run. At least: tokenize, prefill, decode_step, sample, detokenize.

Check

Picket fence = launch tax. Host island = GPU starved by CPU. H2D/D2H on the critical path = accidental copies. .item() = sync cliff. Decode at b=1 is 1+4 until you batch.

Debrief

If the GPU is empty, a faster kernel makes the emptiness larger as a fraction of wall time. Feed the machine first.