Inference practice
M01.02·Accounting·Core·75 minutes·3 min read

View or copy lab

Twenty operations. Predict view vs copy, then write the one-line rule that would have saved the prediction.

Module
PyTorch and the memory wall
Objective
Predict whether a PyTorch op returns a view or allocates, and explain the rule you used.

Do this as a spreadsheet: prediction, rule, then actual after you run (or after you check the answers). Cheating the actual column first trains nothing.

import torch
x = torch.arange(24, dtype=torch.float32).reshape(4, 6)

The twenty

For each, predict view, copy, or sometimes (and the condition).

  1. x[1:]
  2. x[:, ::2]
  3. x.t()
  4. x.T
  5. x.transpose(0, 1)
  6. x.permute(1, 0)
  7. x.reshape(3, 8)
  8. x.reshape(8, 3) after x.t() (i.e. x.t().reshape(8, 3))
  9. x.view(3, 8)
  10. x.view(2, 3, 4)
  11. x.contiguous()
  12. x.t().contiguous()
  13. x.clone()
  14. x.to(torch.float16)
  15. x.to(device=x.device, dtype=x.dtype)
  16. x.float() when x is already float32
  17. x.unsqueeze(0)
  18. x.squeeze()
  19. x.expand(4, 6) (already that shape)
  20. x.expand(8, 4, 6) wait — x.unsqueeze(0).expand(8, 4, 6)

Then the dangerous five

Write one sentence each:

  1. Why reshape is not a synonym of view.
  2. Why x.t().reshape(...) is the copy people ship by accident.
  3. Why expand can make a 1-byte tensor look like a 4 GB tensor without allocating 4 GB.
  4. Why tensor.numpy() can fail after a GPU op even if .cpu() seemed fine — and what copy you just hid.
  5. You need a channel-last activation for a conv. Do you .permute(0,2,3,1) or .contiguous() or both? What did you pay?

Rules

  1. If you have a GPU, run the lab with t.data_ptr() equality and t.is_contiguous().
  2. If you do not, use the check section after you have written predictions.
  3. "I ran it" is not a rule. The rule has to work on the next op.

Acceptance

  1. At least 16/20 predicted correctly before looking.
  2. Items 21–25 are specific: they mention strides, storage, or a byte count, not "it depends" as the whole answer.

Stretch

Implement:

def classify(op_result, parent):
    """Return 'view' | 'copy' using storage pointers, not folklore."""

and run it against the twenty.

Check your work

Views: 1, 2, 3, 4, 5, 6, 7 (contiguous src), 9, 10, 11 (already contiguous → no-op/view of self), 15 (no-op), 16 (no-op), 17, 18, 19, 20 (expand is a view).

Copies: 8 (t().reshape on non-contiguous), 12, 13, 14.

Sometimes: 7 if you had a non-contiguous x; 11 if not contiguous; 15/16 if dtype/device actually change.

21. view refuses if the strides cannot represent the new shape. reshape will copy.

22. Transpose changes strides. reshape wants packed storage, copies the whole tensor.

23. Expand broadcasts strides of 0. nbytes of the view can look huge; storage is the parent.

24. NumPy wants CPU contiguous memory. .cpu() copies device→host; .numpy() on a GPU tensor throws; .cpu().numpy() may still copy if non-contiguous.

25. Permute is a view (wrong strides for most cuDNN paths). .contiguous() (or channels_last memory format) is the copy. You pay the permute's consumer, not the permute.

Debrief

The profiler line aten::copy_ is this list, in production. If you know which ops emit it, you stop being surprised.