Prophet: Fine-grained Load Balancing for Parallel Training of Large-scale MoE Models.
Wei WangZhiquan LaiShengwei LiWeijie LiuKeshi GeYujie LiuAo ShenDongsheng LiPublished in: CLUSTER (2023)
Keyphrases
- fine grained
- load balancing
- coarse grained
- dynamic load balancing
- parallel database systems
- load balance
- parallel query processing
- massively parallel
- load balancing strategy
- distributed systems
- peer to peer
- fault tolerance
- grid computing
- access control
- mobile agents
- low overhead
- load balancing strategies
- round robin
- data partitioning
- data skew
- shared memory
- concurrency control
- data lineage
- high level
- database