Understanding Int4 Quantization for Language Models: Latency Speedup, Composability, and Failure Cases.
Xiaoxia WuCheng LiReza Yazdani AminabadiZhewei YaoYuxiong HePublished in: ICML (2023)
Keyphrases
- language model
- language modeling
- n gram
- document retrieval
- language modelling
- information retrieval
- speech recognition
- retrieval model
- statistical language models
- query expansion
- document ranking
- probabilistic model
- ad hoc information retrieval
- test collection
- language models for information retrieval
- context sensitive
- relevance model
- language model for information retrieval
- word error rate
- vector space model
- smoothing methods
- translation model
- pseudo relevance feedback
- cross lingual
- collaborative filtering