AdaLomo: Low-memory Optimization with Adaptive Learning Rate.

Kai Lv Hang Yan Qipeng Guo Haijun Lv Xipeng Qiu

Published in: ACL (Findings) (2024)

Keyphrases

low memory
adaptive learning rate
globally convergent
learning rate
global optimization
optimization problems
memory requirements
optimization algorithm
constrained optimization
neural network
optimization methods
linear programming
global convergence