MoEBERT: from BERT to Mixture-of-Experts via Importance-Guided Adaptation.

Simiao Zuo Qingru Zhang Chen Liang Pengcheng He Tuo Zhao Weizhu Chen

Published in: NAACL-HLT (2022)

Keyphrases

relative importance
mixture model
neural network
human experts
information retrieval
gaussian distribution
data sets
genetic algorithm
search engine
feature selection
multimedia
multiscale
expectation maximization
logit model