SLoPe: Double-Pruned Sparse Plus Lazy Low-Rank Adapter Pretraining of LLMs.
Mohammad MozaffariAmir YazdanbakhshZhao ZhangMaryam Mehri DehnaviPublished in: CoRR (2024)
Keyphrases
- low rank
- rank minimization
- low rank matrix
- low rank matrices
- low rank subspace
- nuclear norm
- low rank representation
- sparsity constraints
- robust principal component analysis
- linear combination
- kernel matrices
- sparse linear
- regularized regression
- group sparsity
- missing data
- matrix factorization
- convex optimization
- matrix decomposition
- matrix completion
- singular value decomposition
- high order
- low rank approximation
- high dimensional data
- semi supervised
- kernel matrix
- tensor decomposition
- minimization problems
- sparse coding
- high dimensional
- trace norm
- singular values
- dictionary learning
- sparse representation
- tensor factorization
- sparse matrix
- data matrix
- binary matrices
- low rank and sparse
- dimensionality reduction