Compressing Large Language Models using Low Rank and Low Precision Decomposition.
Rajarshi SahaNaomi SaganVarun SrivastavaAndrea J. GoldsmithMert PilanciPublished in: CoRR (2024)
Keyphrases
- language model
- low rank
- language modeling
- tensor decomposition
- matrix factorization
- linear combination
- missing data
- convex optimization
- document retrieval
- singular value decomposition
- n gram
- semi supervised
- probabilistic model
- low rank matrix
- matrix completion
- high order
- trace norm
- high dimensional data
- test collection
- rank minimization
- information retrieval
- retrieval model
- query expansion
- smoothing methods
- language models for information retrieval
- vector space model
- query terms
- higher order
- relevance model
- small number
- similarity measure
- web search
- bayesian networks