Keep the Cost Down: A Review on Methods to Optimize LLM' s KV-Cache Consumption.

Luohe Shi Hongyi Zhang Yao Yao Zuchao Li Hai Zhao

Published in: CoRR (2024)

Keyphrases

information retrieval
probabilistic model
feature selection
optimal solution
preprocessing
empirical studies
benchmark datasets