Pruning Large Language Models to Intra-module Low-rank Architecture with Transitional Activations.
Bowen ShenZheng LinDaren ZhaWei LiuJian LuanBin WangWeiping WangPublished in: ACL (Findings) (2024)
Keyphrases
- language model
- low rank
- language modeling
- matrix factorization
- linear combination
- n gram
- rank minimization
- convex optimization
- missing data
- singular value decomposition
- matrix completion
- document retrieval
- low rank matrix
- semi supervised
- information retrieval
- test collection
- probabilistic model
- retrieval model
- high dimensional data
- smoothing methods
- trace norm
- high order
- query expansion
- collaborative filtering
- vector space model
- image denoising
- relevance model
- query terms
- data sets
- text categorization
- dimensionality reduction
- least squares
- language models for information retrieval