• search
    search
  • reviewers
    reviewers
  • feeds
    feeds
  • assignments
    assignments
  • settings
  • logout

SkipDecode: Autoregressive Skip Decoding with Batching and Caching for Efficient LLM Inference.

Luciano Del CorroAllie Del GiornoSahaj AgarwalBin YuAhmed Hassan AwadallahSubhabrata Mukherjee
Published in: CoRR (2023)
Keyphrases
  • autoregressive
  • random fields
  • moving average
  • non stationary
  • scheduling problem
  • gaussian markov random field
  • information retrieval
  • optical flow
  • least squares