Farewell to Length Extrapolation, a Training-Free Infinite Context with Finite Attention Scope.
Xiaoran LiuQipeng GuoYuerong SongZhigeng LiuKai LvHang YanLinlin LiQun LiuXipeng QiuPublished in: CoRR (2024)
Keyphrases
- training set
- contextual information
- supervised learning
- support vector
- knowledge retrieval
- training examples
- test set
- context sensitive
- context dependent
- finite automata
- finite sets
- real world
- computer software
- focus of attention
- training phase
- context awareness
- training algorithm
- finite number
- training process
- feature space
- search engine