Preble: Efficient Distributed Prompt Scheduling for LLM Serving.
Vikranth SrivatsaZijian HeReyna AbhyankarDongming LiYiying ZhangPublished in: CoRR (2024)
Keyphrases
- lightweight
- meeting scheduling
- scheduling problem
- distributed environment
- multi agent
- distributed systems
- neural network
- cooperative
- multi agent systems
- dynamic scheduling
- cost effective
- tertiary storage
- data transfer
- distributed data
- computer networks
- computationally expensive
- np hard
- special case
- artificial intelligence