蚂蚁 inclusionAI 开源 PolyOCR-Venus：2B/9B OCR 基础模型家族

inclusionAI; PolyOCR-Venus

Repository navigation

NotificationsYou must be signed in to change notification settings; Fork2 (2); Star55 (55)You must be signed in to star a repository

Repository files navigation

README; GPL-2.0 license

for Text-Centric Visual Intelligence

GuangJian Team, Ant Group

Cross-benchmark comparison on OCRBench v2.1, CC-OCR, OmniDocBench v1.6, in-house KIE, and MDPBench. OmniDocBench uses the PolyOCR-9B layout-guided pipeline; MDPBench uses PolyOCR-2.7B. CC-OCR excludes multilingual OCR, and truncated axes are marked. See Benchmark Results for end-to-end scores and protocol details.

PolyOCR-Venus brings text recognition, localization, document parsing, information extraction, multilingual transformation, and OCR-centric reasoning into one instruction-following model family.

Release

2026-09-21: Technical report and OCRBench evaluation code are available in this repository, together with dependency files and a setup guide

Contents

Overview

PolyOCR is a family of 2B and 9B OCR foundation models built on the Qwen3.5 vision-language backbone. A shared instruction-following interface supports tasks ranging from reading individual text regions to recovering document structure and reasoning over visual text.

Unified OCR capabilities. Six capability groups cover text perception, spatial grounding, structured understanding, extraction, transformation, and reasoning across scenes, documents, tables, charts, and diagrams; Large-scale OCR data construction. Curated OCR resources are complemented by controlled synthesis, model-assisted annotation, and quality verification, forming a corpus of approximately 60 million SFT instances; Competence-Guided Policy Optimization (CGPO). Post-training combines verifier-based Group Relative Policy Optimization (GRPO) with adaptive on-policy distillation, routing teacher supervision according to teacher reliability and the teacher–student competence gap

The technical report describes the model family, data construction, training method, and evaluation across OCRBench v2.1, CC-OCR, and OmniDocBench v1.6. The code currently provided here is the OCRBench evaluation module; model weights, training code, and the CC-OCR/OmniDocBench pipelines are not included in this release.

Capabilities

Training Framework

Curriculum supervised fine-tuning. Approximately 60 million instances are organized into three phases. The curriculum gradually shifts emphasis from text perception and document structure toward OCR understanding and reasoning, while retaining general multimodal alignment data; Competence-guided post-training. CGPO uses verifier rewards to estimate student competence and cached teacher generations to estimate teacher competence. A calibrated router controls the distillation weight for each sample; verifier-based GRPO remains active throughout. Both model variants use Qwen3.5-122B-A10B as the teacher for this stage

See the technical report for the data engine, routing formulation, and training settings.

Benchmark Results

Selected end-to-end results from the technical report are shown below. Higher is better in every column.

Layout-guided document parsing. With PP-DocLayoutV3 providing layout regions and reading order, PolyOCR-9B parses cropped text, formula, and table regions and assembles the results into Markdown. This pipeline reaches 95.48 on OmniDocBench v1.6, compared with 91.57 for direct full-page inference.

OCRBench protocol. In this report, OCRBench v2.1 denotes our revision of OCRBench v2: audited and corrected reference annotations together with task-aligned scoring procedures. It preserves the original task coverage. These scores should be compared only with results using the same revised annotations and protocol; original OCRBench v2 scores are not directly comparable. The report appendix documents the changes.

Evaluation

OCRBench evaluation is one component of the PolyOCR project. The implementation is organized under evaluation/ocrbench_v2/ and includes:

PolyOCR two-stage inference configurations: thinking disabled for parsing/recognition and enabled for question answering/reasoning; OpenAI-compatible API inference with resumable runs and prediction export; Scoring for OCR, VQA, document/table/formula parsing, key information extraction, localization, and spotting, including TEDS and CDM; Translation/QA judging, subset rescoring, result merging, and dependency checks

The revised dataset is included locally as evaluation/ocrbench_v2/data/ocrbench_v21.tsv (10,000 samples with base64-embedded images; stored with Git LFS due to its ~1.4 GB size). To reproduce the report's v2.1 protocol, use this dataset together with the matching scoring/judge settings described in the evaluation guide. Model weights and API credentials must be supplied separately.

Installation

Run from the repository root in a Linux environment with Python 3.11 or later:

Full scoring also requires TeX Live components for CDM and NLTK corpora for handwriting metrics. Follow the evaluation guide for data preparation, API configuration, inference, scoring, and dependency checks.

Repository Layout

Citation

If you use PolyOCR or its evaluation protocol in your research, please cite the technical report:

Acknowledgments

PolyOCR builds on Qwen3.5 and uses OCRBench, CC-OCR, and OmniDocBench for evaluation. The included evaluation framework is adapted from VLMEvalKit; its upstream documentation, license, and source manifest are retained with the code.

License

The open-source code in this project is licensed under GPL-2.0. OCRBenchV2.1 is licensed under MIT. Third-party components retain their original licenses; the included VLMEvalKit framework is licensed under Apache-2.0 (see its license).