Transformer Language Models Handle Word Frequency in Prediction Head.
Goro KobayashiTatsuki KuribayashiSho YokoiKentaro InuiPublished in: ACL (Findings) (2023)
Keyphrases
- language model
- word frequency
- language modeling
- n gram
- vector space model
- probabilistic model
- retrieval model
- information retrieval
- document retrieval
- query expansion
- bag of words
- automatic summarization
- relevance model
- text categorization
- document representation
- text classification
- document level
- test collection
- word order
- semantic relatedness
- term frequency
- keywords
- active learning
- text mining
- text retrieval
- query terms
- image classification