Phase diagram of training dynamics in deep neural networks: effect of learning rate, depth, and width.
Dayal Singh KalraMaissam BarkeshliPublished in: CoRR (2023)
Keyphrases
- learning rate
- backpropagation algorithm
- training algorithm
- neural network
- multilayer neural networks
- hidden layer
- activation function
- adaptive learning rate
- feed forward neural networks
- training speed
- feedforward neural networks
- training phase
- convergence rate
- recurrent networks
- learning algorithm
- error function
- training process
- rapid convergence
- convergence speed
- weight vector
- back propagation
- weight update
- artificial neural networks
- multi layer perceptron
- recurrent neural networks
- evolutionary algorithm
- training data
- multi layer
- supervised learning
- genetic programming
- training samples
- depth map
- dynamical systems
- feed forward
- fuzzy neural network