PagedEviction: Structured Block-wise KV Cache Pruning for Efficient Large Language Model Inference
Krishna Teja Chitty-Venkata, Jie Ye, Xian-He Sun, Anthony Kougkas, M. Emani, Venkat Vishwanath, Bogdan Nicolae
arXiv.org