Retrieval-based Knowledge Transfer: An Effective Approach for Extreme Large Language Model Compression.
Jiduan LiuJiahao LiuQifan WangJingang WangXunliang CaiDongyan ZhaoRan Lucien WangRui YanPublished in: CoRR (2023)
Keyphrases
- language model
- knowledge transfer
- retrieval model
- document retrieval
- ad hoc information retrieval
- test collection
- query expansion
- information retrieval
- language modeling
- relevance model
- query terms
- language models for information retrieval
- probabilistic model
- cross language retrieval
- smoothing methods
- n gram
- document length
- knowledge sharing
- statistical language models
- query specific
- document ranking
- transfer learning
- pseudo feedback
- language modelling
- retrieval effectiveness
- cross domain
- mixture model
- jelinek mercer
- pseudo relevance feedback
- context sensitive
- ad hoc retrieval
- term frequency
- learning tasks
- retrieval systems
- statistical language modeling
- maximum likelihood
- information retrieval systems
- relevance feedback
- cross lingual
- translation model
- retrieval accuracy
- image retrieval