Retrieval-based Knowledge Transfer: An Effective Approach for Extreme Large Language Model Compression.
Jiduan LiuJiahao LiuQifan WangJingang WangXunliang CaiDongyan ZhaoRan WangRui YanPublished in: EMNLP (Findings) (2023)
Keyphrases
- language model
- knowledge transfer
- retrieval model
- document retrieval
- ad hoc information retrieval
- test collection
- query expansion
- information retrieval
- language modeling
- language models for information retrieval
- query terms
- knowledge sharing
- transfer learning
- smoothing methods
- cross language retrieval
- n gram
- language modelling
- query specific
- statistical language models
- document ranking
- document length
- mixture model
- probabilistic model
- relevance model
- cross domain
- learning tasks
- jelinek mercer
- retrieval effectiveness
- context sensitive
- statistical language modeling
- term frequency
- image retrieval
- retrieval accuracy
- pairwise
- pseudo relevance feedback
- similarity measure
- active learning
- data mining
- feature selection
- pseudo feedback
- information extraction
- relevance feedback
- markov random field
- translation model