Baidu Open-Sources Unlimited OCR 2, a 3.34B Model Supporting Parsing of Hundreds of Pages of Documents
Baidu has open-sourced Unlimited OCR 2, a 3.34B-parameter vision-language model that can parse hundreds of pages of books, financial reports, and reports in a single request. The weights are released under the MIT license, and the code is on GitHub. Its core change is SR-SWA sparse attention, which compresses text and visual attention into a fixed 512-token reference set, keeping the latency and memory of each decoding step essentially flat as documents grow longer. Baidu self-reports that the model scores 84.6 on MPDocBench and 77.1 on MarathonDocBench, where the next-best model Kimi-K3 scores 33.7, though its heading-hierarchy sub-item is only 48.2.
Why read this
AI score 64
The original text presents SR-SWA's fixed 512-token reference set mechanism and long-document benchmark comparisons, which can be used to assess the attention cost trade-offs of long-document OCR.
Topics
Body
Add as a preferredsource on Google
Baidu has released Unlimited OCR 2, an open-source OCR model built for very long documents. The 3B model parses books, filings and reports of hundreds of pages in one request. Its core change is SR-SWA, a sparse attention scheme that keeps per-step decoding cost nearly flat. Weights ship under the MIT license, with code on GitHub.
TL;DR
Size: 3.34B parameters. Active parameters not disclosed. Max position length 131,072 tokens.
Runs on: 1 CUDA GPU in BF16. Baidu tested on an NVIDIA A800-80GB. BF16 weights total about 6.7 GB.
Performance: first place on both multi-page benchmarks Baidu tested, against 9 other models (self-reported).
Best: 77.1 overall on MarathonDocBench (hundreds of pages). The next model, Kimi-K3, scores 33.7.
Worst: 48.2 on heading hierarchy (TEDS) on MarathonDocBench, its lowest sub-score.
What is Unlimited OCR 2?
Unlimited OCR 2 is a 3B vision-language model from Baidu that turns ordered page images into structured text. Output covers text, tables, formulas, figures, headings and reading order. It succeeds Unlimited-OCR, released in June 2026 to push DeepSeek-OCR further.
TEXTTABLELAYOUTIs your PDF parser actually getting it right?omni_parse_bench · scores text, tables & layout · open sourceStar on GitHub→Sponsored
The config keeps a DeepSeek-OCR style layout. A SAM ViT-B plus CLIP-L encoder feeds a 12-layer mixture-of-experts decoder. Each token routes to 6 of 64 experts, plus 2 shared experts. The DeepSeek-OCR paper describes this decoder as DeepSeek3B-MoE-A570M. Baidu does not state an active count for version 2. In multi-page mode, each page costs 273 image tokens.
How does SR-SWA work?
Full attention cost grows quadratically with sequence length. During decoding, each new token scans a longer KV cache. At 273 image tokens per page, 100 pages add 27,300 input tokens before any output.
SR-SWA stands for Sparse Reference Sliding Window Attention. It compresses text-visual attention into a fixed set of working references. Baidu’s model card gives 512 tokens as the example size. The config pairs a 128-token local window with top-16 selection over 32-token blocks. That is 16 blocks of 32 tokens, matching the 512-token budget.
This keeps per-step latency and memory nearly constant as documents grow. It calls the result ‘unlimited-horizon’ parsing within a finite context window. Image encoding and total runtime still rise with page count.
How does it score on MPDocBench and MarathonDocBench?
Baidu team tests on MPDocBench, with dozens of pages per document, and MarathonDocBench, with hundreds. MPDocBench-Parse is a public benchmark of 433 documents and 3,246 pages. We could not find a public MarathonDocBench release.
Per Baidu’s leaderboard chart, Unlimited OCR 2 scores 84.6 on MPDocBench. Qwen3.8-Max follows at 78.3, then Qwen-3.8-Flash-Next at 77.4. On MarathonDocBench it scores 77.1. Kimi-K3 is next at 33.7, then Gemini-3.8-Flash at 28.6.
The drop between benchmarks is the real story. Qwen3.8-Max falls from 78.3 to 22.9 on hundred-page documents. Unlimited OCR 2 falls 7.5 points. Its predecessor, Unlimited-OCR, scores 0.0 on MarathonDocBench.
Sub-scores (MPDocBench / MarathonDocBench):
Heading hierarchy: 60.1 / 48.2
Single-metric margins on MPDocBench are thin. Qwen3.8-Max scores 94.3 on text and 89.2 on formulas. All figures are self-reported and appear only in a chart image.
Comparison with other open OCR models
Unlimited OCR 2Unlimited-OCRDeepSeek-OCR-2PaddleOCR-VL-1.6
DeveloperBaiduBaiduDeepSeekBaidu PaddlePaddle
Parameters3.34B3.34B3.39B0.96B
Active parametersNot disclosedNot disclosedNot disclosedNot disclosed
LicenseMITMITApache 2.0Apache 2.0
Paper / releaseOct 2026 (report)Jun 2026 (arXiv)Jan 2026 (arXiv)Jun 2026 (arXiv)
Long-document mechanismSR-SWA, 512-token reference setMulti-page infer_multi, no SR-SWANot disclosedNot disclosed
MPDocBench overall84.6 (Baidu)61.9 (Baidu)Not reportedNot reported
MarathonDocBench overall77.1 (Baidu)0.0 (Baidu)Not reportedNot reported
Parameter counts and licenses from each model’s Hugging Face page. Benchmark scores from Baidu’s chart.
How do you run Unlimited OCR 2?
The model card lists Transformers for quick tests and SGLang for serving. In Transformers, infer handles 1 page with dynamic tiling, called gundam mode. infer_multi parses every page of a document in 1 request, so the model keeps cross-page context. Loading requires trust_remote_code=True, which runs Python code from the repo.
SGLang serving uses a customized wheel from Baidu, not upstream SGLang. Baidu’s example schedules 10 pages per window and reuses the previous window’s KV tail. Smaller windows cut peak memory but weaken cross-page merging. Baidu warns a default bias setting can trigger a 462 MiB allocation and run out of memory.
Key Takeaways
Unlimited OCR 2 is a 3.34B, MIT-licensed OCR model from Baidu.
SR-SWA caps attention at a fixed 512-token reference set.
It scores 77.1 on MarathonDocBench, against 33.7 for the next model.
Heading hierarchy is the weak spot, at 48.2 on long documents.
All benchmark numbers are Baidu’s own.
Asif RazzaqWebsite | + postsBio ⮌
Asif RazzaqNace AI Releases Drex for Search for AI Agents: 92.05% on BrowseComp-Plus With 12.6 Search Calls
Asif RazzaqNanbeige Releases Nanbeige4.3-33B, a 33B Open Agentic Coding Model
$uv pip install omni-parse-bench✓text✓tables✓layout→ scored per pageStar on GitHub→Sponsored
RELATED ARTICLESMORE FROM AUTHOR
OpenAI Launches GPT-6.1 Terra, Halving GPT-6.1 Sol’s API Price for Agent Work
Nace AI Releases Drex for Search for AI Agents: 92.05% on BrowseComp-Plus With 12.6 Search Calls
Nanbeige Releases Nanbeige4.3-33B, a 33B Open Agentic Coding Model
Researchers from Stanford and UC Berkeley Introduce TokenGate: Token-Gated Attention Makes Long-Context Prefill Up to 11.3x Faster