LoRETTA: Low-Rank Economic Tensor-Train Adaptation for Ultra-Low-Parameter Fine-Tuning of Large Language Models.
Yifan YangJiajun ZhouNgai WongZheng ZhangPublished in: CoRR (2024)
Keyphrases
- fine tuning
- language model
- low rank
- trace norm
- high order
- frobenius norm
- language modeling
- tensor decomposition
- linear combination
- convex optimization
- matrix completion
- matrix factorization
- missing data
- information retrieval
- probabilistic model
- rank minimization
- low rank matrix
- n gram
- singular value decomposition
- high dimensional data
- test collection
- semi supervised
- retrieval model
- query expansion
- pairwise
- higher order
- singular values
- vector space model
- tensor factorization
- machine learning
- collaborative filtering
- data analysis
- smoothing methods
- high dimensional
- data mining