Phase diagram of early training dynamics in deep neural networks: effect of the learning rate, depth, and width.
Dayal Singh KalraMaissam BarkeshliPublished in: NeurIPS (2023)
Keyphrases
- learning rate
- backpropagation algorithm
- training algorithm
- neural network
- multilayer neural networks
- activation function
- training speed
- adaptive learning rate
- hidden layer
- feed forward neural networks
- learning algorithm
- convergence rate
- feedforward neural networks
- training phase
- error function
- recurrent networks
- training process
- rapid convergence
- back propagation
- convergence speed
- recurrent neural networks
- dynamical systems
- supervised learning
- weight vector
- weight update
- multi layer
- deep architectures
- multi layer perceptron
- radial basis function
- convergence theorem
- rbf neural network
- machine learning