Statistical Perspective of Top-K Sparse Softmax Gating Mixture of Experts.

Huy Nguyen Pedram Akbarian Fanqi Yan Nhat Ho

Published in: ICLR (2024)

Keyphrases

high dimensional
statistical analysis
query processing
compressive sensing
sparse data
neural network
data driven
statistical models
highly relevant
block max
sparse matrix
skyline queries
information theoretic
user defined
linear combination
mixture model
language model
domain knowledge