Measure the memory and latency budget
Measure the memory and latency budget with explicit assumptions, intermediate steps and a bounded conclusion.
Text lesson
MLE PathChecking your access…
Also in Technical membership bundle, Career membership bundle, ML Residency bundle, LLMs A to Z learning bundle.
Shared topics and concepts are exploration links, not universal prerequisites.
26 of 26 modules
Measure the memory and latency budget with explicit assumptions, intermediate steps and a bounded conclusion.
Text lesson
Reason about training memory and IO costs with explicit assumptions, intermediate steps and a bounded conclusion.
Text lesson
Account for the main training-memory components with explicit assumptions, intermediate steps and a bounded conclusion.
Text lesson
Distinguish frozen parameters from absent parameter storage with explicit assumptions, intermediate steps and a bounded conclusion.
Text lesson
Exchange saved activations for recomputation with explicit assumptions, intermediate steps and a bounded conclusion.
Text lesson
Bound whole-system gains from a faster component with explicit assumptions, intermediate steps and a bounded conclusion.
Text lesson
Trace dynamic and continuous batching decisions with explicit assumptions, intermediate steps and a bounded conclusion.
Text lesson
Trace compiler representations and transformation obligations with explicit assumptions, intermediate steps and a bounded conclusion.
Text lesson
Preserve an objective across data-parallel workers with explicit assumptions, intermediate steps and a bounded conclusion.
Text lesson
Trace expert dispatch, capacity and imbalance with explicit assumptions, intermediate steps and a bounded conclusion.
Text lesson
Preserve the intended gradient when accumulating microbatches with explicit assumptions, intermediate steps and a bounded conclusion.
Text lesson
Match communication costs to the actual device path with explicit assumptions, intermediate steps and a bounded conclusion.
Text lesson
Separate compilation, specialization and execution costs with explicit assumptions, intermediate steps and a bounded conclusion.
Text lesson
Budget and map key-value cache storage with explicit assumptions, intermediate steps and a bounded conclusion.
Text lesson
Track storage, scope and movement through GPU memory with explicit assumptions, intermediate steps and a bounded conclusion.
Text lesson
Separate storage precision, compute precision and scaling with explicit assumptions, intermediate steps and a bounded conclusion.
Text lesson
Map GPU work and inspect coalesced accesses with explicit assumptions, intermediate steps and a bounded conclusion.
Text lesson
Budget replicated and sharded model states with explicit assumptions, intermediate steps and a bounded conclusion.
Text lesson
Distinguish instruction programmability, configured logic and specialized circuits. Select a feasible candidate using end-to-end constraints and measured cost.
Text lesson
Use arithmetic intensity to bound a kernel with explicit assumptions, intermediate steps and a bounded conclusion.
Text lesson
Measure LLM serving latency, tails and throughput with explicit assumptions, intermediate steps and a bounded conclusion.
Text lesson
Partition a matrix operation without changing its meaning with explicit assumptions, intermediate steps and a bounded conclusion.
Text lesson
Evaluate tiling and fusion with explicit traffic ledgers with explicit assumptions, intermediate steps and a bounded conclusion.
Text lesson
Trace Triton block indexing and tail safety with explicit assumptions, intermediate steps and a bounded conclusion.
Text lesson