Inference

CS336 Notes: Lecture 10 - Inference

LLM inference optimization: understanding the prefill vs decode split, KV cache management, speculative decoding, and why inference is fundamentally memory-bound.