SpecInfer: Accelerating Generative LLM Serving with Speculative Inference and Token Tree Verification.
Xupeng MiaoGabriele OliaroZhihao ZhangXinhao ChengZeyu WangRae Ying Yee WongZhuoming ChenDaiyaan ArfeenReyna AbhyankarZhihao JiaPublished in: CoRR (2023)
Keyphrases
- tree structure
- probabilistic inference
- clique tree
- tree grammars
- model checking
- generative model
- tree search
- markov logic networks
- formal verification
- tree structures
- bayesian networks
- signature verification
- inference process
- bayesian inference
- index structure
- unsupervised learning
- knowledge base
- spanning tree
- probabilistic model
- undirected graphical models
- tree models
- tree construction
- inference mechanism
- graphical models
- maximum likelihood
- grammatical inference
- exact inference
- belief networks
- formal methods
- probabilistic reasoning