BERTology Meets Biology: Interpreting Attention in Protein Language Models.
Jesse VigAli MadaniLav R. VarshneyCaiming XiongRichard SocherNazneen Fatema RajaniPublished in: ICLR (2021)
Keyphrases
- language model
- language modeling
- protein structure prediction
- document retrieval
- n gram
- probabilistic model
- retrieval model
- information retrieval
- test collection
- statistical language models
- query expansion
- language modelling
- speech recognition
- context sensitive
- protein sequences
- smoothing methods
- protein structure
- vector space model
- amino acids
- translation model
- language models for information retrieval
- document ranking
- language model for information retrieval
- ad hoc information retrieval
- pseudo relevance feedback
- query terms
- cross language information retrieval
- word error rate
- okapi bm
- relevance feedback
- decision trees
- search engine