Inference Infrastructure15 min
Distributing a Model Across GPUs
Compare data, tensor, pipeline, and context parallelism using memory, communication, and failure boundaries.
Huang Tzu Lin
TagBlog
#Distributed Inference
2 posts
Inference Infrastructure14 min
Deploying Mixture-of-Experts Models
Separate model routing from expert placement, and calculate the memory and communication each layout needs.
Huang Tzu Lin