Is attention required for ICL? Exploring the Relationship Between Model Architecture and In-Context Learning Ability.
Ivan LeeNan JiangTaylor Berg-KirkpatrickPublished in: ICLR (2024)
Keyphrases
- computational model
- learning process
- formal model
- learning systems
- learning mechanism
- conceptual model
- learning algorithm
- context dependent
- statistical model
- management system
- probability distribution
- generative model
- probabilistic model
- user model
- prior knowledge
- multi layer
- automatically learned
- learned models
- real time
- reference model
- learning scheme
- web services
- learning models
- neural nets
- visual attention
- learning tasks
- experimental data