Sparse Attention with Linear Units.

Biao Zhang Ivan Titov Rico Sennrich

Published in: CoRR (2021)

Keyphrases

high dimensional
visual attention
simple linear
regularized least squares
linear complexity
sparse linear
real time
neural network
information retrieval
decision trees
multiscale
closed form
linear model
sparse data
focus of attention