SLTrain: a sparse plus low-rank approach for parameter and memory efficient pretraining.
Andi HanJiaxiang LiWei HuangMingyi HongAkiko TakedaPratik JawanpuriaBamdev MishraPublished in: CoRR (2024)
Keyphrases
- memory efficient
- low rank
- low rank matrix
- rank minimization
- low rank matrices
- low rank subspace
- nuclear norm
- sparsity constraints
- robust principal component analysis
- low rank representation
- missing data
- group sparsity
- kernel matrices
- sparse linear
- matrix decomposition
- regularized regression
- convex optimization
- matrix factorization
- matrix completion
- low rank approximation
- linear combination
- singular value decomposition
- high order
- tensor decomposition
- kernel matrix
- semi supervised
- high dimensional data
- data matrix
- sparse matrix
- singular values
- high dimensional
- trace norm
- binary matrices
- minimization problems
- data mining
- interior point methods
- low rank and sparse
- dimensionality reduction
- small number