Inference Infrastructure14 min
Separating Prefill and Decode
Decide when phase separation is worth the cost of KV transfer, extra coordination, and a more complex deployment.
Huang Tzu Lin
TagBlog
#Prefill-Decode
1 post
1 post
Decide when phase separation is worth the cost of KV transfer, extra coordination, and a more complex deployment.