AI 热点

RecSA以对数查询成本将32K训练外推至2M tokens

UCSD Hao AI Lab更新于 11月19日 09:02

RecSA: Million-Token Extrapolation at Logarithmic Cost 🚀

AI 导读

RecSA通过在KV cache上构建树状索引,将每次查询成本控制为O(log L),使仅在32K序列上训练的Qwen3-4B模型外推至2M tokens。该模型使用0.62B tokens训练,在RULER的1M长度上得分60.5,强于Dense + DCA的40.0。在一张H200上,RecSA于524K长度将预填充和解码分别加速3.2×和3.8×;博客:https://t.co/yKpRitz8zw,论文:https://t.co/cWiIZS40Cy,代码:https://t.co/qAFANlalhH。

推荐理由

AI 评分 55

该研究把KV cache组织成分层树,并以评测和延迟数据呈现32K训练外推至2M上下文的表现。

主题

标签

正文

RecSA: Million-Token Extrapolation at Logarithmic Cost 🚀

RecSA builds a tree over the KV cache and lets each query search it level by level, so the cost per query grows as O(log L) and a model trained only at 32K extrapolates to 2M tokens.

Initialized from Qwen3-4B and trained on 0.62B tokens, all at 32K, RecSA outperforms all baselines on RULER from 131K to 2M, scoring 60.5 at 1M against 40.0 for the strongest baseline (Dense + DCA). On one H200, it speeds up prefill by 3.2× and decoding by 3.8× at 524K.

Blog: https://t.co/yKpRitz8zw Paper: https://t.co/cWiIZS40Cy Code: https://t.co/qAFANlalhH

来源

UCSD Hao AI Lab