Understand this story
Researchers from Tsinghua University and the Chinese University of Hong Kong have open-sourced AKV, a KV cache engineering serving system. Built on SGLang, the system lets agents directly discard selected KV cache entries and reuse the rest, rather than summarizing old history. Across 3 models and 4 agent benchmarks, average cost dropped by 27.7%, and average task scores improved by 1.7 points. The code is released under the Apache-2.0 license and currently supports only the Qwen, GPT-OSS, and MiniMax-M2 text models.
AI-generated from public reports
Latest development · Report
Tsinghua and CUHK Open-Source AKV: A KV Cache Engineering Service System for LLM AgentsDevelopments
1 confirmed developments
- Source:MarkTechPost
Reports and discussion timeline
Browse reports and discussion by time. Their evidence levels remain distinct.
Oct 26
MarkTechPost
Tsinghua and CUHK Open-Source AKV: A KV Cache Engineering Service System for LLM Agents
Researchers from Tsinghua University and the Chinese University of Hong Kong have open-sourced AKV, a KV cache engineering service system built on SGLang that lets agents directly discard selected KV cache entries and reuse the rest, rather than summarizing old history. Across 3 models and 4 agent benchmarks, average cost dropped by 27.7% and average task scores improved by 1.7 points. The code is released under the Apache-2.0 license and currently supports only the Qwen, GPT-OSS, and MiniMax-M2 text models.