GaLore: Memory-Efficient LLM Training by Gradient Low-Rank Projection.
Jiawei ZhaoZhenyu ZhangBeidi ChenZhangyang WangAnima AnandkumarYuandong TianPublished in: CoRR (2024)
Keyphrases
- low rank
- memory efficient
- matrix factorization
- convex optimization
- missing data
- singular value decomposition
- high dimensional data
- linear combination
- low rank matrix
- matrix completion
- semi supervised
- rank minimization
- matrix decomposition
- trace norm
- singular values
- high order
- robust principal component analysis
- training set
- kernel matrix
- minimization problems
- low rank matrices
- neural network
- regularized regression
- higher order
- supervised learning
- stochastic gradient descent
- background subtraction
- training samples
- natural images
- collaborative filtering
- non rigid structure from motion
- small number
- pattern recognition