Data Distributional Properties Drive Emergent In-Context Learning in Transformers.
Stephanie ChanAdam SantoroAndrew K. LampinenJane WangAaditya SinghPierre H. RichemondJames L. McClellandFelix HillPublished in: NeurIPS (2022)
Keyphrases
- data sets
- high quality
- learning algorithm
- database
- prior knowledge
- raw data
- background knowledge
- data structure
- supervised learning
- missing data
- data sources
- training data
- data mining
- neural network
- data analysis
- original data
- synthetic data
- learning tasks
- data quality
- relational databases
- databases
- human experts
- data distribution
- experimental data
- domain experts
- learning systems
- computer systems
- knowledge acquisition
- small number