Opening the Black Box: Analyzing Attention Weights and Hidden States in Pre-trained Language Models for Non-language Tasks.
Mohamad BalloutUlf KrumnackGunther HeidemannKai-Uwe KühnbergerPublished in: xAI (3) (2023)
Keyphrases
- black box
- language model
- pre trained
- language modeling
- hidden states
- hidden markov models
- speech recognition
- document retrieval
- information retrieval
- probabilistic model
- n gram
- test cases
- mixture model
- natural language
- reinforcement learning
- training data
- generative model
- training samples
- computer vision
- machine learning