Baidu Open-Sources Unlimited OCR 2: 3B/A0.5B MoE Architecture Supports Ultra-Long Document Parsing
Baidu has open-sourced Unlimited OCR 2, an OCR and layout parsing model designed for documents ranging from dozens to hundreds of pages, adopting a 3B/A0.5B MoE architecture. Compared with v1's R-SWA, v2 upgrades the decoder attention to SR-SWA and adds Unlimited-Horizon Decoding, which compresses the attention interaction between text and vision into a fixed-capacity working reference set (the README gives 512 tokens as an example), and then combines this with page-by-page chunked decoding, enabling ultra-long documents to be parsed within a limited context. The authors claim that at the hundred-page scale, step-by-step decoding speed is approximately 7.0× that of Full Attention and approximately 5.6× that of R-SWA, with latency and GPU memory barely increasing as documents grow longer.
Why read this
AI score 61
The original text explains the attention mechanism changes in v2 relative to v1 and the speed data at the hundred-page scale, which can be used to assess the engineering trade-offs for ultra-long document parsing.
Topics
View original postUnlimited OCR 2 is here, and it's open source. 📖 ✨ Longer documents can mean a lot more attention computation. Our new SR-SWA mechanism keeps that in check by condensing text-visual attention interactions into a fixed-size set of working references. Here's what it delivers: - Document parsing at virtually any length, within a finite context window - Nearly constant per-step decoding latency and memory footprint as documents grow longer - State-of-the-art performance on MPDocBench and MarathonDocBench, covering documents from dozens to hundreds of pages Check out the model here ↓
View original
Unlimited OCR 2 is here, and it's open source. 📖 ✨ Longer documents can mean a lot more attention computation. Our new SR-SWA mechanism keeps that in check by condensing text-visual attention interactions into a fixed-size set of working references. Here's what it delivers: - Document parsing at virtually any length, within a finite context window - Nearly constant per-step decoding latency and memory footprint as documents grow longer - State-of-the-art performance on MPDocBench and MarathonDocBench, covering documents from dozens to hundreds of pages Check out the model here ↓
Body · AI translation
Baidu open-sources Unlimited OCR 2: ultra-long document parsing, with attention cost kept from growing with page count as much as possible
Unlimited OCR 2 is an open-source OCR / layout parsing model aimed at documents of "tens to hundreds of pages," with a 3B/A0.5B MoE model architecture: it uses SR-SWA to compress text–vision attention into a fixed-size working reference set, and combined with page-chunked decoding, it lets ultra-long documents be processed within a limited context, and it claims that step-by-step decoding latency and GPU memory barely grow as documents get longer.
Open-source model: https://t.co/Z8uP95MZ1Y
What changed relative to v1 v1 (R-SWA) could already do multi-page end-to-end parsing, but the paper states that input was roughly capped at 40 pages, and for that reason it was not included on MarathonDocBench. v2 upgrades the decoder attention to SR-SWA (Sparse Reference Sliding Window Attention) and adds Unlimited-Horizon Decoding; the backbone is still the DeepEncoder + MoE decoder line, retrained from the v1 weights. The slogan moves from "long-range" to "unlimited-range."
What SR-SWA is doing (per the technical report) Long documents make text–vision attention very expensive. R-SWA kept local interactions in check, but distant visual tokens still grew with page count. SR-SWA's approach is: compress the interaction into a fixed-capacity working reference (the README gives 512 tokens as an example). Roughly two steps: Page Gate first narrows candidates by page; Token Sieve then scores summaries of chunks, using α-entmax sparse routing to keep only the top-k distant chunks plus the local window. On the decoding side, it chunks by page, overlaps page boundaries by 1 page and reuses the KV tail, avoiding GPU memory blowup at ultra-long lengths. The authors claim that at the hundred-page scale, step-by-step speed is about 7.0× that of Full Attention and about 5.6× that of R-SWA.