Don't Decay the Learning Rate, Increase the Batch Size.

Samuel L. Smith Pieter-Jan Kindermans Quoc V. Le

Published in: CoRR (2017)

Keyphrases

learning rate
batch size
learning algorithm
convergence rate
batch mode
poisson process
convergence speed
single item
batch processing
machine learning
pairwise
active learning
cost function
delta bar delta