The continued usefulness of vocabulary tests for evaluating large language models.
Gonzalo MartínezJavier CondeElena Merino GómezBeatriz Bermúdez-MargarettoJosé Alberto HernándezPedro ReviriegoMarc BrysbaertPublished in: CoRR (2023)
Keyphrases
- language model
- language modeling
- out of vocabulary
- spoken term detection
- speech recognition
- probabilistic model
- n gram
- document retrieval
- retrieval model
- query expansion
- test collection
- language modelling
- information retrieval
- ad hoc information retrieval
- context sensitive
- statistical language models
- smoothing methods
- vector space model
- query terms
- document ranking
- language models for information retrieval
- word error rate
- pseudo relevance feedback
- okapi bm
- broadcast news
- term dependencies
- translation model
- relevance model
- error rate
- document length
- feature selection
- machine learning