Fiddler: CPU-GPU Orchestration for Fast Inference of Mixture-of-Experts Models.

Keisuke Kamahori Yile Gu Kan Zhu Baris Kasikci

Published in: CoRR (2024)

Keyphrases

random fields
real time
probabilistic model
statistical model
statistical models
probabilistic inference
parallel implementation