#LLM Serving

Measure useful capacity with a frozen workload, per-class latency goals, queue evidence, and failure tests.

Explore token-budget scheduling, chunked prefill, preemption, and backpressure in a small runnable simulation.

Trace a request through prefill, decode, caching, and streaming—and reconcile its timing and memory costs.