Scan and Snap: Understanding Training Dynamics and Token Composition in 1-layer Transformer.

Yuandong Tian Yiping Wang Beidi Chen Simon S. Du

Published in: CoRR (2023)

Keyphrases

recurrent networks
multi layer
feed forward neural networks
training set
online learning
training examples
training phase
dynamic model
database
neural network
knowledge base
video sequences
hidden markov models
subject matter
upper layer