Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolation.

Ofir Press Noah A. Smith Mike Lewis

Published in: CoRR (2021)

Keyphrases

visual attention
linear systems
database
neural network
piecewise linear
search engine
video sequences
input data
test cases
fixed length
focus of attention
total length