Low-Rank Bottleneck in Multi-head Attention Models.

Srinadh Bhojanapalli Chulhee Yun Ankit Singh Rawat Sashank J. Reddi Sanjiv Kumar

Published in: ICML (2020)

Keyphrases

low rank
probabilistic model
convex optimization
missing data
rank minimization
matrix decomposition
linear combination
singular value decomposition
kernel matrix
data mining
high order
visual attention
semi supervised
data analysis
nonnegative matrix factorization
trace norm
image segmentation
decision trees