Layer-wise Pruning of Transformer Attention Heads for Efficient Language Modeling.

Kyuhong Shim Iksoo Choi Wonyong Sung Jungwook Choi

Published in: CoRR (2021)

Keyphrases

language modeling
language model
retrieval model
information retrieval
query expansion
probabilistic model
n gram
cross lingual
information retrieval systems
document retrieval
relevance model
trec collections
sentence retrieval