Convergence of ADAM with Constant Step Size in Non-Convex Settings: A Simple Proof.
Alokendu MazumderBhartendu KumarManan TayalPunit RathorePublished in: CoRR (2023)
Keyphrases
- step size
- convergence rate
- convergence speed
- faster convergence
- variable step size
- line search
- hessian matrix
- neural network
- evolutionary programming
- adaptive filter
- gradient method
- global optimum
- steepest descent method
- risk minimization
- differential evolution
- cost function
- optimization algorithm
- face recognition
- genetic algorithm