ROME: Memorization Insights from Text, Probability and Hidden State in Large Language Models.
Bo LiQinghua ZhaoLijie WenPublished in: CoRR (2024)
Keyphrases
- language model
- hidden state
- information retrieval
- language modeling
- markov models
- hidden markov models
- document retrieval
- n gram
- reinforcement learning
- speech recognition
- document level
- probabilistic model
- retrieval model
- text retrieval
- test collection
- statistical language models
- language modelling
- dynamical systems
- query expansion
- web documents
- text mining
- relevance model
- smoothing methods
- query terms
- vector space model
- transition probabilities
- partially observable
- probability distribution
- keywords
- conditional probabilities
- document ranking
- text documents
- language models for information retrieval
- pseudo relevance feedback
- partially observable markov decision processes
- translation model
- machine learning
- pairwise
- okapi bm