Stepping on the Edge: Curvature Aware Learning Rate Tuners.
Vincent RouletAtish AgarwalaJean-Bastien GrillGrzegorz SwirszczMathieu BlondelFabian PedregosaPublished in: CoRR (2024)
Keyphrases
- learning rate
- convergence rate
- learning algorithm
- high curvature
- convergence speed
- multilayer neural networks
- edge detection
- hidden layer
- rapid convergence
- multiscale
- adaptive learning rate
- error function
- intensity profile
- training algorithm
- convergence theorem
- weight vector
- zero crossing
- natural gradient
- edge detector
- scale space
- reinforcement learning
- delta bar delta