PiSSA: Principal Singular Values and Singular Vectors Adaptation of Large Language Models.

Fanxu Meng Zhaohui Wang Muhan Zhang

Published in: CoRR (2024)

Keyphrases

language model
singular values
singular vectors
singular value decomposition
document retrieval
probabilistic model
n gram
information retrieval
low rank
retrieval model
fundamental matrix
query expansion
vector space model
dimensionality reduction
missing data
least squares
semi supervised
pairwise
image processing