mL-BFGS: A Momentum-based L-BFGS for Distributed Large-scale Neural Network Optimization.
Yue NiuZalan FabianSunwoo LeeMahdi SoltanolkotabiSalman AvestimehrPublished in: Trans. Mach. Learn. Res. (2023)
Keyphrases
- neural network
- stochastic gradient descent
- learning rate
- convergence rate
- limited memory
- global convergence
- quasi newton
- superlinear convergence
- quasi newton method
- maximum likelihood
- artificial neural networks
- distributed environment
- distributed systems
- optimization algorithm
- small scale
- line search
- image reconstruction from projections
- real world
- training algorithm
- peer to peer
- loss function
- optimization method
- genetic algorithm
- distributed stream processing
- least squares
- optimization problems
- back propagation
- data intensive
- activation function
- neural network model
- optimization methods
- cooperative
- high scalability
- optimization process
- convergence speed
- risk minimization
- recurrent neural networks
- highly non linear
- feed forward
- feedforward neural networks
- step size
- differential evolution
- sliding window
- neural network training
- weight vector
- evolutionary algorithm
- self organizing maps