Efficient Adaptive Batching of DNN Inference Services for Improved Latency.

Osama Khan Junyeol Yu Yeonjae Kim Euiseong Seo

Published in: ICOIN (2024)

Keyphrases

web services
service oriented
service composition
context aware
single machine
ubiquitous computing
cost efficient
end users
probability distribution
data management
computationally efficient
prefetching
service oriented architecture
service discovery
low overhead