Cross Entropy of Neural Language Models at Infinity - A New Bound of the Entropy Rate.
Shuntaro TakahashiKumiko Tanaka-IshiiPublished in: Entropy (2018)
Keyphrases
- cross entropy
- language model
- language modeling
- retrieval model
- n gram
- information retrieval
- probabilistic model
- document retrieval
- network architecture
- query expansion
- log likelihood
- test collection
- information theoretic
- neural network
- maximum likelihood
- mixture model
- relevance model
- smoothing methods
- language models for information retrieval
- artificial neural networks
- decision trees
- evaluation metrics
- web search engines
- mutual information
- feature selection