LoRA-FA: Memory-efficient Low-rank Adaptation for Large Language Models Fine-tuning.
Longteng ZhangLin ZhangShaohuai ShiXiaowen ChuBo LiPublished in: CoRR (2023)
Keyphrases
- fine tuning
- language model
- memory efficient
- low rank
- agent technology
- language modeling
- convex optimization
- linear combination
- matrix factorization
- singular value decomposition
- low rank matrix
- n gram
- probabilistic model
- missing data
- semi supervised
- document retrieval
- retrieval model
- rank minimization
- matrix completion
- high order
- query expansion
- information retrieval
- test collection
- high dimensional data
- software agents
- trace norm
- language models for information retrieval
- smoothing methods
- multi agent systems
- query terms
- vector space model
- mobile agents
- generative model
- knn
- data analysis
- pattern recognition
- data sets