AI Trends

Trending story

RecSA Achieves Logarithmic Query Costs with a KV Cache Tree

1 reports1 story updatesUpdated Nov 19 03:20

Understand this story

Story summary

The English summary is being updated. See the reporting timeline below.

Latest development · Report

UCSD Hao AI Lab Releases RecSA, Enabling Million-Token Context Extrapolation at Logarithmic Cost

Developments

1 confirmed developments

  1. English translation is being prepared.

    Source:UCSD Hao AI Lab

Reports and discussion timeline

Browse reports and discussion by time. Their evidence levels remain distinct.

Nov 19

  1. UCSD Hao AI LabFirst-party

    UCSD Hao AI Lab Releases RecSA, Enabling Million-Token Context Extrapolation at Logarithmic Cost

    RecSA, proposed by UCSD Hao AI Lab, builds a tree structure over the KV cache, allowing the cost of each query to grow as O(log L) and enabling Qwen3-4B, trained only on 32K sequences, to extrapolate to 2M tokens. RecSA was trained on 0.62B tokens and outperformed the baselines on RULER at every length from 131K to 2M, scoring 60.5 at 1M versus 40.0 for the strongest baseline, Dense + DCA. On a single H200, RecSA accelerated prefill by 3.2× and decoding by 3.8× at 524K. The code has been open-sourced and includes Triton and CUDA kernels as well as evaluation implementations.

Related stories

  1. Related storiesRecSA Outperforms All Baselines in the RULER Long-Context Benchmark
  2. Related storiesRecSA Outperforms All Baselines in the RULER Evaluation for Ultra-Long Contexts
  3. Related storiesRecSA Achieves Extrapolation to 2M Tokens with Logarithmic Query Cost
  4. Related storiesRecSA Extrapolates Context to 2 Million Tokens with Logarithmic Query Cost