Dynamic Low-rank Estimation for Transformer-based Language Models.

Ting Hua Xiao Li Shangqian Gao Yen-Chang Hsu Yilin Shen Hongxia Jin

Published in: EMNLP (Findings) (2023)

Keyphrases

language model
low rank
language modeling
matrix factorization
missing data
convex optimization
probabilistic model
document retrieval

retrieval model
n gram
linear combination
rank minimization
information retrieval
query expansion
low rank matrix

semi supervised
test collection
singular value decomposition
matrix completion
high dimensional data
smoothing methods
collaborative filtering

relevance model
vector space model
singular values
markov random field
information retrieval systems
query terms
regularized regression