A view of mini-batch SGD via generating functions: conditions of convergence, phase transitions, benefit from negative momenta.
Maksim VelikanovDenis KuznedelevDmitry YarotskyPublished in: ICLR (2023)
Keyphrases
- phase transition
- random constraint satisfaction problems
- constraint satisfaction
- satisfiability problem
- randomly generated
- np complete
- hard problems
- combinatorial problems
- sufficient conditions
- random instances
- random graphs
- graph coloring
- stochastic local search
- boolean satisfiability
- np complete problems
- convergence rate
- branch and bound algorithm
- convergence speed
- learning algorithm
- sat problem
- search algorithm