Transformers learn to implement preconditioned gradient descent for in-context learning.

Kwangjun Ahn Xiang Cheng Hadi Daneshmand Suvrit Sra

Published in: CoRR (2023)

Keyphrases

learning algorithm
unsupervised learning
active learning
learning contexts
reinforcement learning
learning process
supervised learning
learning scheme
learning scenarios
learning systems
iterative methods
efficient learning
context dependent
learning tasks
background knowledge
prior knowledge
e learning