A kernel has no single speed

Use the honest timer from the sibling drill. If you do not have a GPU, compute the shape of the story from the table below and write the same sentences.

Part A — The sweep

For square GEMM in fp16, FLOPs = 2 n³. Time the median of 50 reps after 20 warmup, n ∈ {256, 512, 1024, 2048, 4096, 8192} if memory allows.

Fill:

n	median ms	TFLOP/s	% of a 312 TFLOP peak

If you are on paper, use this synthetic (but typical) A100-ish table and still do the arithmetic:

n	median ms
256	0.042
512	0.051
1024	0.089
2048	0.31
4096	2.05
8192	14.8

Part B — Three sentences you must be able to say

1. At which n is time almost independent of n? What regime is that?
2. At which n does TFLOP/s stop climbing fast? What does that imply?
3. Replace this sentence with a correct one: "cuBLAS matmul takes 2 ms on A100."

Part C — The add, same sizes

Time x + x at the same n. Report GB/s. Does TFLOP/s of the add ever look "good"? Should it?

Acceptance

1. TFLOP/s at 256 is an order of magnitude below 4096.
2. You named overhead, then a ridge crossing or plateau.
3. You refused a shape-free milliseconds claim.
4. The add is a bandwidth story; quoting TFLOP/s for it is a category error.

Stretch

Add torch.compile on a tiny elementwise chain at n=256 and n=4096. Warm up 30 times. Where does compile win, and where is it a rounding error?

Check (synthetic table)

FLOPs(256)=3.36e7 → 0.80 TFLOP/s. FLOPs(4096)=1.37e11 → 67 TFLOP/s. FLOPs(8192)=1.10e12 → 74 TFLOP/s. Time barely moves 256→512: overhead. Plateau 4096→8192: you are seeing the card, not the launch. "2 ms" is n=4096 in this table and a different universe at n=512.

Debrief

A kernel does not have a speed. It has a speed at a size, a dtype, a layout, and a batch. The timer drill taught you how to measure. This drill teaches you what is allowed to be called a measurement.