Shortcut-connected Expert Parallelism for Accelerating Mixture-of-Experts.
Weilin CaiJuyong JiangLe QinJunwei CuiSunghun KimJiayi HuangPublished in: CoRR (2024)
Keyphrases
- domain experts
- expert advice
- human experts
- mixture model
- parallel processing
- domain knowledge
- subject matter experts
- expert knowledge
- shared memory
- gaussian mixture model
- connected components
- knowledge acquisition
- knowledge engineers
- hidden markov models
- massively parallel
- parallel architectures
- expert systems
- data sets