In-Context Pretraining: Language Modeling Beyond Document Boundaries.
Weijia ShiSewon MinMaria LomeliChunting ZhouMargaret LiXi Victoria LinNoah A. SmithLuke ZettlemoyerScott YihMike LewisPublished in: CoRR (2023)
Keyphrases
- language modeling
- language model
- information retrieval
- language modeling approaches
- document length
- retrieval model
- improvements in retrieval effectiveness
- term weighting schemes
- document retrieval
- probabilistic model
- query expansion
- cross lingual
- relevance model
- pseudo feedback
- context sensitive
- n gram
- term weighting
- information retrieval systems
- query specific
- ad hoc information retrieval
- vector space model
- document ranking
- retrieval systems
- text classification
- document language models
- language modeling framework
- term dependencies
- web documents
- query terms
- low dimensional
- tf idf
- query processing
- data mining
- metadata
- retrieval effectiveness