Stochasticity of Deterministic Gradient Descent: Large Learning Rate for Multiscale Objective Function.
Lingkai KongMolei TaoPublished in: CoRR (2020)
Keyphrases
- learning rate
- objective function
- multiscale
- error function
- update rule
- convergence rate
- natural gradient
- cost function
- learning algorithm
- weight vector
- multilayer neural networks
- convergence speed
- hidden layer
- image processing
- optimization problems
- multi objective
- global optimum
- rapid convergence
- optimal solution
- adaptive learning rate
- image segmentation
- linear programming
- activation function
- training algorithm
- machine learning
- convergence theorem
- deterministic annealing
- constrained optimization
- denoising
- back propagation
- linear combination
- bp neural network algorithm
- delta bar delta