SmartPipe: Intelligently Freezing Layers in Pipeline Parallelism for Distributed DNN Training.
Nadia NiknamiAbdalaziz SawwanJie WuPublished in: ICPADS (2023)
Keyphrases
- training process
- distributed systems
- computer networks
- multi agent
- peer to peer
- parallel processing
- parallel execution
- training phase
- multi layer
- parallel computation
- lightweight
- training set
- communication overhead
- feed forward neural networks
- real time
- massively parallel
- processing pipeline
- computational power
- agent technology
- training algorithm
- computing environments
- fault tolerant
- test set
- training examples
- multi agent systems