Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolation.

Ofir Press Noah Smith Mike Lewis

Published in: ICLR (2022)

Keyphrases

focus of attention
input data
real time
data mining
information retrieval
closed form
visual attention
statistical tests
linear model
fixed length
linear complexity
total length
shift register