Transformer Language Models Handle Word Frequency in Prediction Head.
Goro KobayashiTatsuki KuribayashiSho YokoiKentaro InuiPublished in: CoRR (2023)
Keyphrases
- language model
- word frequency
- language modeling
- retrieval model
- n gram
- vector space model
- probabilistic model
- document retrieval
- information retrieval
- term frequency
- query terms
- test collection
- query expansion
- automatic summarization
- bag of words
- relevance model
- retrieval effectiveness
- feature selection
- text retrieval
- document representation
- ad hoc retrieval