Hydra: Multi-head low-rank adaptation for parameter efficient fine-tuning.
Sanghyeon KimHyunmo YangYunghyun KimYoungjoon HongEunbyung ParkPublished in: Neural Networks (2024)
Keyphrases
- fine tuning
- low rank
- missing data
- matrix factorization
- convex optimization
- linear combination
- singular value decomposition
- data sets
- low rank matrix
- matrix completion
- rank minimization
- matrix decomposition
- high dimensional data
- robust principal component analysis
- trace norm
- kernel matrix
- semi supervised
- data matrix
- dimensionality reduction
- higher order
- small number
- learning algorithm