Baidu Open-Sources Unlimited OCR 2: 3B/A0.5B MoE Architecture Supports Ultra-Long Document Parsing
English text is being prepared.
Baidu has open-sourced Unlimited OCR 2, an OCR and layout parsing model designed for documents ranging from dozens to hundreds of pages, adopting a 3B/A0.5B MoE architecture. Compared with v1's R-SWA, v2 upgrades the decoder attention to SR-SWA and adds Unlimited-Horizon Decoding, which compresses the attention interaction between text and vision into a fixed-capacity working reference set (the README gives 512 tokens as an example), and then combines this with page-by-page chunked decoding, enabling ultra-long documents to be parsed within a limited context. The authors claim that at the hundred-page scale, step-by-step decoding speed is approximately 7.0× that of Full Attention and approximately 5.6× that of R-SWA, with latency and GPU memory barely increasing as documents grow longer.
Why read this
AI score 61
The original text explains the attention mechanism changes in v2 relative to v1 and the speed data at the hundred-page scale, which can be used to assess the engineering trade-offs for ultra-long document parsing.
Topics
View original postUnlimited OCR 2 is here, and it's open source. 📖 ✨ Longer documents can mean a lot more attention computation. Our new SR-SWA mechanism keeps that in check by condensing text-visual attention interactions into a fixed-size set of working references. Here's what it delivers: - Document parsing at virtually any length, within a finite context window - Nearly constant per-step decoding latency and memory footprint as documents grow longer - State-of-the-art performance on MPDocBench and MarathonDocBench, covering documents from dozens to hundreds of pages Check out the model here ↓
View original
Unlimited OCR 2 is here, and it's open source. 📖 ✨ Longer documents can mean a lot more attention computation. Our new SR-SWA mechanism keeps that in check by condensing text-visual attention interactions into a fixed-size set of working references. Here's what it delivers: - Document parsing at virtually any length, within a finite context window - Nearly constant per-step decoding latency and memory footprint as documents grow longer - State-of-the-art performance on MPDocBench and MarathonDocBench, covering documents from dozens to hundreds of pages Check out the model here ↓
Body
百度开源 Unlimited OCR 2：超长文档解析，注意力开销尽量不随页数涨
Unlimited OCR 2 是面向「几十到几百页」文档的开源 OCR / 版面解析模型，3B/A0.5B 的 MoE 模型架构：用 SR-SWA 把文本–视觉注意力压到固定大小的工作参考集，再配合按页分块解码，让超长文档能在有限上下文里跑完，并声称逐步解码的延迟和显存几乎不随文档变长而涨。
开源模型：https://t.co/Z8uP95MZ1Y
相对 v1 改了什么 v1（R-SWA）已能做多页端到端解析，但论文写明输入大约卡在 40 页，也因此没上 MarathonDocBench。v2 把解码器注意力升到 SR-SWA（Sparse Reference Sliding Window Attention），并加上 Unlimited-Horizon Decoding；骨干仍是 DeepEncoder + MoE 解码器一路，从 v1 权重再训。口号从「长程」推到「不限程」。
SR-SWA 在做什么（技术报告口径） 长文档会让文本–视觉注意力很贵。R-SWA 管住了局部交互，但远端视觉 token 仍会随页数涨。SR-SWA 的做法是：把交互压进固定容量的工作参考（README 举例 512 tokens）。大致两步：Page Gate 先按页收窄候选；Token Sieve 再对块做摘要打分，用 α-entmax 稀疏路由，只保留 top-k 远端块加局部窗口。解码侧按页切块，页边界重叠 1 页并复用 KV 尾部，避免超长时显存爆掉。作者称百页量级上，逐步速度相对 Full Attention 约 7.0×、相对 R-SWA 约 5.6×。