Towards GPU Memory Efficiency for Distributed Training at Scale.
Runxiang ChengChris CaiSelman YilmazRahul MitraMalay BagMrinmoy GhoshTianyin XuPublished in: SoCC (2023)
Keyphrases
- data transfer
- memory usage
- distributed environment
- dynamically created
- training set
- high scalability
- multi agent
- computer networks
- real time
- peer to peer
- computational complexity
- scale space
- training process
- training algorithm
- training phase
- data structure
- distributed systems
- data sets
- online learning
- lightweight
- test set
- main memory
- memory requirements
- associative memory
- computing power