SpecTr: Fast Speculative Decoding via Optimal Transport.

Ziteng Sun Ananda Theertha Suresh Jae Hun Ro Ahmad Beirami Himanshu Jain Felix X. Yu

Published in: NeurIPS (2023)

Keyphrases

dynamic programming
worst case
closed form
data sets
decoding algorithm
machine learning
similarity measure
multi agent
lower bound
probabilistic model
state space
optimal strategy