Convergence proof for stochastic gradient descent in the training of deep neural networks with ReLU activation for constant target functions.
Martin HutzenthalerArnulf JentzenKatharina PohlAdrian RiekertLuca ScarpaPublished in: CoRR (2021)
Keyphrases
- stochastic gradient descent
- stochastic gradient
- convergence proof
- neural network
- loss function
- least squares
- matrix factorization
- step size
- random forests
- support vector machine
- online algorithms
- regularization parameter
- multiple kernel learning
- importance sampling
- weight vector
- monte carlo
- learning algorithm
- machine learning