Practice Ladder
The course teaches the model. The ladder is the hours. Every open lesson has drills that ask for an artifact: a byte ledger, a benchmark, a profiler report, a capacity plan, a scheduler simulation, or an incident playbook.
Do them with a timer, but do not optimise for speed. Optimise for a checkable chain of reasoning: prediction, measurement, explanation, and the next action.
M00
Orientation
4 drills · ~5 hours, reading and drills
01
Accounting
Regime from the symptoms
Eight production stories. For each one, name the regime, name the lever that would actually help, and name the lever that would waste the week.
Lesson · The three regimes
02
Accounting
Ridge-point workbook
Compute ridge points for four cards, place six kernels on each roofline, and write the sentence that decides what you are allowed to optimise.
Lesson · The three regimes
03
Accounting
Decode floor calculator
Compute the bandwidth floor on tokens/s for four model-card pairs, then explain why a measured number that beats the floor is a measurement bug.
Lesson · Why inference is asymmetric
04
Serving
Prefill or decode: the incident
Product says the model is slow. Five tickets. Split TTFT from ITL, name the phase, and refuse a fix that targets the other one.
Lesson · Why inference is asymmetric
M01
PyTorch and the memory wall
8 drills · ~8 hours, notebooks and drills
01
Accounting
Tensor Memory Accountant
Turn three preprocessing blocks into byte ledgers: storage, views, copies, and the line that actually hurts.
Lesson · Tensors and the bytes they move
02
Accounting
View or copy lab
Twenty operations. Predict view vs copy, then write the one-line rule that would have saved the prediction.
Lesson · Tensors and the bytes they move
03
Benchmarking
Honest GPU Timer
Build a tiny benchmarking harness that warms up, uses CUDA events, repeats, reports medians, and refuses to benchmark CPU fallbacks by accident.
Lesson · Timing a GPU without lying to yourself
04
Benchmarking
A kernel has no single speed
Sweep matmul size, convert ms into TFLOP/s, and write the sentence that forbids quoting a kernel speed without a shape.
Lesson · Timing a GPU without lying to yourself
05
Accounting
Roofline Classifier
Classify toy kernels as memory-bound, compute-bound, or overhead-bound from flops, bytes, time, and hardware ceilings.
Lesson · Arithmetic intensity, measured
06
Accounting
Decode intensity from scratch
Derive arithmetic intensity of a decode step from parameter count and dtype, then place it on four ridges without looking at a blog.
Lesson · Arithmetic intensity, measured
07
Accounting
Fusion byte ledger
Count round trips for an eager elementwise chain, predict the fused traffic, then explain a speedup in bytes rather than in adjectives.
Lesson · Fusion, and the traffic you did not need
08
Benchmarking
Overhead-bound collapse
A size sweep where fusion's 4× becomes 1.05×. Explain the shape in launches and bytes, then say what compile / graphs are for.
Lesson · Fusion, and the traffic you did not need
M02
See inside the box
4 drills · ~8 hours, drills
01
Profiling
Profiler Triage Report
Turn an operator table into a concise performance report with suspects, evidence, and the next experiment.
Lesson · The first profiler pass
02
Profiling
Named ranges and the five-line report
Design profiler ranges for a prefill+decode request, then write the five-line report from a table that only makes sense with those names.
Lesson · The first profiler pass
03
Profiling
Timeline Idle Gap Detective
Given a toy timeline, identify whether the GPU is waiting on launch overhead, host work, synchronization, or data transfer.
Lesson · Reading the GPU timeline
04
Profiling
Four timeline smells
Four traces. Name the idle pattern, name the cause class, and name the first experiment that would shrink the white space.
Lesson · Reading the GPU timeline
M03
The GPU execution model
2 drills · ~6 hours, reading and drills
01
Kernels
Occupancy is not the goal
Three kernels with occupancy, bytes, and time. Decide whether raising occupancy is the lever, a sideshow, or active harm.
Lesson · The GPU execution model without mysticism
02
Kernels
Coalescing and the transpose tax
A free transpose, a slow consumer, and a contiguous copy that may or may not be worth it. Account for the extra segments.
Lesson · The GPU execution model without mysticism
M05
Serving an LLM
4 drills · ~8 hours, reading and drills
01
Serving
KV Cache Capacity Planner
Compute live-token capacity for an LLM deployment and turn VRAM into max concurrency under different prompt and output lengths.
Lesson · KV cache math
02
Serving
GQA and the live-token budget
Recompute a capacity plan after cutting KV heads, then write the admission rule in live tokens — not in 'batch size'.
Lesson · KV cache math
03
Systems
Continuous Batching Simulator
Write a small scheduler simulation that admits new requests between decode steps and compares it to static batching.
Lesson · Continuous batching and the serving scheduler
04
Operations
Serving SLO Playbook
Design the measurement and rollout plan for an inference endpoint with real latency targets, capacity limits, and rollback rules.
Lesson · Continuous batching and the serving scheduler