PipeFisher: Efficient Training of Large Language Models Using Pipelining and Fisher Information Matrices.

Kazuki Osawa Shigang Li Torsten Hoefler

Published in: CoRR (2022)

Keyphrases

language model
fisher information
language modeling
n gram
probabilistic model
document retrieval
retrieval model
information retrieval
test collection
mixture model
query expansion
vector space model
positive definite
training set