Contrastive Distillation on Intermediate Representations for Language Model Compression.
Siqi SunZhe GanYuwei FangYu ChengShuohang WangJingjing LiuPublished in: EMNLP (1) (2020)
Keyphrases
- language model
- intermediate representations
- language modeling
- n gram
- probabilistic model
- document retrieval
- information retrieval
- intermediate representation
- structured learning
- retrieval model
- high level
- query expansion
- smoothing methods
- mixture model
- test collection
- context sensitive
- ad hoc information retrieval
- translation model
- pairwise
- word clouds
- maximum likelihood
- multi class
- image features
- natural language
- search engine