Transformers Learn Higher-Order Optimization Methods for In-Context Learning: A Study with Linear Models.

Deqing Fu Tian-Qi Chen Robin Jia Vatsal Sharan

Published in: CoRR (2023)

Keyphrases

optimization methods
linear models
higher order
learning algorithm
continuous optimization
optimization problems
incremental learning
simulated annealing
super resolution
particle swarm optimization
unsupervised learning
optimization method
stochastic methods