FireRedTeam 发布 FireRed-OCR，OmniDocBench v1.5 得分 92.94%

FireRedTeam; FireRed-OCR

Repository navigation

NotificationsYou must be signed in to change notification settings; Fork12 (12); Star296 (296)You must be signed in to star a repository

Repository files navigation

README; Apache-2.0 license

Figure 1: Performance comparison on the OmniDocBench v1.5 benchmark. FireRed-OCR achieves state-of-the-art performance among end-to-end solutions, ranking first with a score above 92%.

🔥 FireRed-OCR

FireRed-OCR is a systematic framework designed to specialize general Large Vision-Language Models (LVLMs) into high-performance, pixel-precise structural document parsing experts.

General VLMs frequently suffer from "Structural Hallucination" (e.g., disordered rows, invented formulas) when processing complex documents. FireRed-OCR addresses this by shifting the paradigm from "impressionist" text generation to "structural engineering," achieving State-of-the-Art (SOTA) results on authoritative benchmarks like OmniDocBench v1.5.

✨ Key Features

SOTA Performance: Achieves 92.94% overall score on OmniDocBench v1.5, significantly outperforming DeepSeek-OCR 2, OCRVerse, and massive general VLMs (e.g., Gemini-3.0 Pro，Qwen3-VL-235B); Structural Integrity: Utilizing Format-Constrained GRPO (Group Relative Policy Optimization), the model enforces strict syntactic validity, eliminating common errors like unclosed tables or invalid LaTeX formulas; "Geometry + Semantics" Data Factory: A novel data engine that uses geometric feature clustering and multi-dimensional tagging to synthesize balanced datasets, effectively handling long-tail layouts; Progressive Training Pipeline:

Multi-task Pre-alignment: Establishes spatial grounding; Specialized SFT: Standardizes full-image Markdown output; Format-Constrained GRPO: Self-correction via Reinforcement Learning; In-the-Wild Robustness: Demonstrates superior resilience on complex, non-standard layouts (FireRedBench) compared to traditional pipeline systems like PaddleOCR

2026.02.28: We released FireRed-OCR-2B weights. Check more details in the Model Zoo section

🗂️ Model Zoo

🏗️ Model Architecture

The FireRed-OCR framework transforms a general VLM into a structural expert through a three-stage progressive training strategy:

Stage 1: Multi-task Pre-alignment: Trains the model on detection, region recognition, and layout-to-markdown tasks to ground visual perception; Stage 2: Specialized SFT: Fine-tunes on a high-quality, standardized Markdown dataset to ensure logical consistency and hierarchical expression; Stage 3: Format-Constrained GRPO: Applies Reinforcement Learning with specific rewards for Formula Syntax, Table Integrity, Hierarchical Closure, and Text Accuracy

⚡️ Quick Start

FireRed-OCR is based on the Qwen3-VL architecture. You can use the following code snippets to generate structured Markdown from document images.

We evaluate FireRed-OCR on OmniDocBench v1.5 and FireRedBench.

For PaddleOCR-VL-1.5 and GLM-OCR on OCRBench, scores are reported as API / pure VLM.

📜 License Agreement

The code and the weights of FireRed-OCR are licensed under Apache 2.0.

🖊️ Citation

We kindly encourage citation of our work if you find it useful.

⚠️ Ethics Statement

FireRed-OCR is a technical tool designed for document digitization and structural parsing.

Prohibited Use: This project must not be used to generate or process content that is illegal, defamatory, pornographic, harmful, or that violates the privacy, rights, or interests of individuals or organizations; User Responsibility: Users are solely responsible for any content generated using this project. The authors and contributors assume no responsibility or liability for any misuse of the codebase or for any consequences resulting from its use

🤝 Acknowledgements

We would like to thank the developers of the amazing open-source projects, including Qwen-VL, PaddleOCR, olmOCR and the broader OCR community.