Inference Infrastructure14 min
Deploying Mixture-of-Experts Models
Separate model routing from expert placement, and calculate the memory and communication each layout needs.
Huang Tzu Lin
TagBlog
#Expert Parallelism
1 post
1 post
Separate model routing from expert placement, and calculate the memory and communication each layout needs.