AI Trends

Trending story

RecSA Outperforms All Baselines in the RULER Evaluation for Ultra-Long Contexts

1 reports1 story updatesUpdated Nov 19 03:20

Understand this story

Story summary

The English summary is being updated. See the reporting timeline below.

Latest development · Report

RecSA Research Scales a Model Trained on 32K to 2M Tokens at Logarithmic Cost

Developments

1 confirmed developments

  1. English translation is being prepared.

    Source:UCSD Hao AI Lab

Reports and discussion timeline

Browse reports and discussion by time. Their evidence levels remain distinct.

Nov 19

  1. UCSD Hao AI LabFirst-party

    RecSA Research Scales a Model Trained on 32K to 2M Tokens at Logarithmic Cost

    RecSA from UCSD Hao AI Lab builds a tree structure over the KV cache, making the cost of each query grow as O(log L) and enabling a model trained only on 32K contexts to extrapolate to 2M tokens. Initialized from Qwen3-4B and trained on 0.62B tokens, the method scores 60.5 on RULER's 1M test, compared with 40.0 for the strong Dense + DCA baseline. On a single H200, it speeds up 524K prefill and decoding by 3.2× and 3.8×, respectively; the code is open source and includes kernels, Qwen3 integration, RULER evaluation, and latency benchmarks: https://t.co/qAFANlalhH

Related stories

  1. Related storiesRecSA Achieves Logarithmic Query Costs with a KV Cache Tree
  2. Related storiesRecSA Outperforms All Baselines in the RULER Long-Context Benchmark
  3. Related storiesRecSA Achieves Extrapolation to 2M Tokens with Logarithmic Query Cost
  4. Related storiesRecSA Extrapolates Context to 2 Million Tokens with Logarithmic Query Cost