Throughput Maximization of DNN Inference: Batching or Multi-Tenancy?

Seyed Morteza Nabavinejad Masoumeh Ebrahimi Sherief Reda

Published in: CoRR (2023)

Keyphrases

bayesian inference
response time
scheduling problem
probabilistic inference
objective function
real time
neural network
bayesian networks
inference engine
single machine
sensor networks
inference mechanism
efficient learning
probabilistic reasoning
training process
lower bound
knowledge base
search engine
data sets