ASVD: Activation-aware Singular Value Decomposition for Compressing Large Language Models.
Zhihang YuanYuzhang ShangYue SongQiang WuYan YanGuangyu SunPublished in: CoRR (2023)
Keyphrases
- singular value decomposition
- language model
- language modeling
- singular values
- document retrieval
- low rank
- n gram
- probabilistic model
- least squares
- information retrieval
- query expansion
- statistical language models
- smoothing methods
- retrieval model
- latent semantic indexing
- test collection
- dimensionality reduction
- principal component analysis
- vector space model
- text categorization
- query terms
- pattern recognition
- bag of words
- high dimensional
- relevance model
- translation model
- low rank approximation
- feature selection
- machine learning