OwLore: Outlier-weighed Layerwise Sampled Low-Rank Projection for Memory-Efficient LLM Fine-tuning.
Pengxiang LiLu YinXiaowei GaoShiwei LiuPublished in: CoRR (2024)
Keyphrases
- memory efficient
- fine tuning
- low rank
- convex optimization
- matrix factorization
- linear combination
- missing data
- low rank matrix
- singular value decomposition
- rank minimization
- high dimensional data
- fine tuned
- regularized regression
- matrix completion
- high order
- matrix decomposition
- semi supervised
- kernel matrix
- minimization problems
- non rigid structure from motion
- low rank matrices
- data matrix
- small number
- rank constraint
- low rank representation
- least squares
- neural network
- incomplete data
- sparse representation
- pairwise