LoRETTA: Low-Rank Economic Tensor-Train Adaptation for Ultra-Low-Parameter Fine-Tuning of Large Language Models.
Yifan YangJiajun ZhouNgai WongZheng ZhangPublished in: NAACL-HLT (2024)
Keyphrases
- fine tuning
- language model
- low rank
- trace norm
- high order
- frobenius norm
- tensor decomposition
- language modeling
- n gram
- linear combination
- matrix factorization
- convex optimization
- missing data
- probabilistic model
- low rank matrix
- high dimensional data
- matrix completion
- singular value decomposition
- information retrieval
- retrieval model
- test collection
- rank minimization
- query expansion
- semi supervised
- vector space model
- higher order
- smoothing methods
- dimensionality reduction
- pairwise
- small number
- multi task
- collaborative filtering
- information retrieval systems