Inference Infrastructure21 min
Benchmarking Inference Capacity
Measure useful capacity with a frozen workload, per-class latency goals, queue evidence, and failure tests.
Huang Tzu Lin
TagBlog
#Benchmarking
1 post
1 post
Measure useful capacity with a frozen workload, per-class latency goals, queue evidence, and failure tests.