Inference Infrastructure18 min
Prefill-Decode Disaggregation: Splitting the Two Stages of Inference
Post I-00 established that LLM inference has two phases with fundamentally different resource profiles. Prefill processes all input tokens in parallel and is compute-bound -- the GPU's arithmetic units are the bottleneck. Decode generates tokens one at a time and is memory-bandwidth-bound -- the ...
Huang Tzu Lin