Attention is not all you need: pure attention loses rank doubly exponentially with depth.

Yihe Dong Jean-Baptiste Cordonnier Andreas Loukas

Published in: ICML (2021)

Keyphrases

search engine
website
search algorithm
database
neural network
information retrieval
decision making
wide range
support vector
object recognition
visual attention
selective attention