What Kinds of Tokens Benefit from Distant Text? An Analysis on Long Context Language Modeling.

Yutong Hu Quzhe Huang Kangcheng Luo Yansong Feng

Published in: CoRR (2024)

Keyphrases

language modeling
language model
information retrieval
retrieval model
query expansion
statistical language models
learning algorithm
context sensitive