Attention is Not All You Need: Pure Attention Loses Rank Doubly Exponentially with Depth.

Yihe Dong Jean-Baptiste Cordonnier Andreas Loukas

Published in: CoRR (2021)

Keyphrases

wide range
artificial intelligence
neural network
super resolution
visual attention
real time
real world
information retrieval
information systems
three dimensional