Accelerating LLM Inference with Staged Speculative Decoding.

Benjamin Spector Chris Re

Published in: CoRR (2023)

Keyphrases

bayesian networks
probabilistic inference
decoding algorithm
belief networks
random fields
bayesian inference
inference engine
efficient learning
inference process
soft decision
database
decoding process
inference mechanism
bayesian model
probabilistic reasoning
social networks
data sets