Bridging Information-Theoretic and Geometric Compression in Language Models.
Emily ChengCorentin KervadecMarco BaroniPublished in: CoRR (2023)
Keyphrases
- information theoretic
- language model
- mutual information
- information theory
- language modeling
- speech recognition
- document retrieval
- n gram
- information bottleneck
- information retrieval
- theoretic framework
- probabilistic model
- retrieval model
- context sensitive
- test collection
- statistical language models
- query expansion
- jensen shannon divergence
- language modelling
- entropy measure
- mixture model
- information theoretic measures
- language models for information retrieval
- relevance model
- kullback leibler divergence
- query terms
- geometric structure
- translation model
- pseudo relevance feedback
- vector space model
- image registration
- document ranking