Login / Signup
NACL: A General and Effective KV Cache Eviction Framework for LLM at Inference Time.
Yilong Chen
Guoxia Wang
Junyuan Shang
Shiyao Cui
Zhenyu Zhang
Tingwen Liu
Shuohuan Wang
Yu Sun
Dianhai Yu
Hua Wu
Published in:
ACL (1) (2024)
Keyphrases
</>
main contribution
lightweight
inference process
data sets
high quality
special case
theoretical framework
closely related
general theory
bayesian networks
bayesian framework
unified framework