Revisiting Gradient Clipping: Stochastic bias and tight convergence guarantees.
Anastasia KoloskovaHadrien HendrikxSebastian U. StichPublished in: ICML (2023)
Keyphrases
- stochastic approximation
- convergence proof
- upper bound
- lower bound
- convergence speed
- convergence rate
- worst case
- volume rendering
- monte carlo
- gradient direction
- initial conditions
- trade off
- global convergence
- stochastic optimization
- image processing
- stochastic nature
- decision trees
- gradient field
- stochastic programming
- gradient information
- iterative algorithms
- neural network
- image sequences
- edge detection
- reinforcement learning