Mutual Information Constraints for Monte-Carlo Objectives to Prevent Posterior Collapse Especially in Language Modelling.
Gábor MelisAndrás GyörgyPhil BlunsomPublished in: J. Mach. Learn. Res. (2022)
Keyphrases
- monte carlo
- language modelling
- mutual information
- language model
- markov chain monte carlo
- n gram
- markov chain
- monte carlo simulation
- importance sampling
- monte carlo tree search
- adaptive sampling
- similarity measure
- pseudo relevance feedback
- particle filter
- ad hoc retrieval
- image registration
- probability distribution
- feature selection
- matrix inversion
- markovian decision
- posterior distribution
- posterior probability
- document ranking
- variance reduction
- query expansion
- relevance feedback
- probabilistic model
- data mining