Distributed Training of Deep Neuronal Networks: Theoretical and Practical Limits of Parallel Scalability.
Janis KeuperPublished in: CoRR (2016)
Keyphrases
- map reduce
- parallel database systems
- load balance
- distributed processing
- distributed systems
- fault tolerant
- fault tolerance
- real world
- cooperative
- scalable distributed
- master slave
- deep architectures
- neuronal networks
- theoretical considerations
- high scalability
- load balancing
- distributed environment
- pc cluster
- multi agent
- parallel execution
- deep learning
- training samples
- theoretical analysis
- single processor
- parallel computation
- distributed data
- training phase
- supervised learning
- theoretical underpinnings
- communication cost
- message passing
- parallel processing
- data sets
- lightweight
- commodity hardware
- fully distributed
- training examples
- test set
- parallel computing
- parallel implementation
- distributed computing
- computer networks