Understand this story
The English summary is being updated. See the reporting timeline below.
Latest development · Report
RecSA Research Scales a Model Trained on 32K to 2M Tokens at Logarithmic CostDevelopments
1 confirmed developments
- Source:UCSD Hao AI Lab
Reports and discussion timeline
Browse reports and discussion by time. Their evidence levels remain distinct.
Nov 19
UCSD Hao AI LabFirst-party
RecSA Research Scales a Model Trained on 32K to 2M Tokens at Logarithmic Cost
RecSA from UCSD Hao AI Lab builds a tree structure over the KV cache, making the cost of each query grow as O(log L) and enabling a model trained only on 32K contexts to extrapolate to 2M tokens. Initialized from Qwen3-4B and trained on 0.62B tokens, the method scores 60.5 on RULER's 1M test, compared with 40.0 for the strong Dense + DCA baseline. On a single H200, it speeds up 524K prefill and decoding by 3.2× and 3.8×, respectively; the code is open source and includes kernels, Qwen3 integration, RULER evaluation, and latency benchmarks: https://t.co/qAFANlalhH
AI Trends