Communication trade-offs for synchronized distributed SGD with large step size.
Kumar Kshitij PatelAymeric DieuleveutPublished in: CoRR (2019)
Keyphrases
- step size
- stochastic gradient descent
- trade off
- communication overhead
- cost function
- convergence rate
- communication cost
- convergence speed
- evolutionary programming
- faster convergence
- distributed control
- hessian matrix
- gradient method
- variable step size
- steepest descent method
- line search
- distributed systems
- adaptive filter
- neural network
- random forests
- genetic algorithm ga
- back propagation
- support vector
- image segmentation
- genetic algorithm