1.3B Open-Source Document Parsing Model OXR Tops Three Leaderboards, Introducing RSI Self-Evolution Training

QbitAI | WeChat Official Account QbitAI

No way, RSI applied to OCR is this powerful??

Recently, a domestic open-source model with only 1.3B parameters has crushed document parsing.

It ranks first on all three benchmarks: OmniDocBench, olmOCR-Bench, and DocParsingBench.

On a single RTX PRO 6000, it completes end-to-end parsing of 100 pages of documents in just 101 seconds.

Damn, Anthropic's efficiency at distilling paper books is about to take off now (just kidding).

It's also blowing up in overseas developer communities, having already gained 1.1K+ GitHub Stars.

RSI.

This new self-evolving OCR species is called OXR.

Optical "Everything" Recognition.

Document parsing, a mountain that sounds like it has no technical content, yet remains unconquered to this day.

Traditional OCR, when faced with a seemingly ordinary PDF, is often basically a disaster scene.

Two-column papers get read in series, handwritten formulas are recognized as garbled text, large cross-page tables have their structure collapse outright, and as for chemical structure formulas, many tools simply discard them as messy lines...

Once this kind of low-quality extraction is thrown into RAG, no matter how smart the model is, all downstream Tool Calls will fail.

An open-source document parsing model called OXR has topped all three world-class benchmarks.

Moreover, the entire evaluation pipeline and code are fully open-source and reproducible.

I also gave it a quick try myself, and it really is different. Finally, it's not an artificial idiot anymore.

And OXR can swallow 13 types of document elements in one go:

Tables, formulas, code blocks, images, chemical equations, footnotes, headers, footers, stamps...

Applied in specific scenarios, it looks like this —

Magazines: outputs structured MD that preserves complete semantic structure and reading order.

Mathematical formulas: directly converted into standard LaTeX code.

Papers?

Don't make the LLM slowly call tools to read them either; two-column layouts don't affect quality.

It can even parse math exam papers???

Looks like GPT will be able to better help elementary school students finish their summer homework in the future o(╥﹏╥)o.

Of course, the most exciting thing is still the open-source part.

You should know that the significance of open-sourcing this kind of vertical small model is very different from that of LLMs.

To be honest, although open-source LLMs are great, they basically have little to do with ordinary users.

Even for a model with only 27B parameters, local deployment costs are absurdly high—at minimum you'd need an M5 Pro 64G, and that's without considering output speed.

But this OXR thing is only 1.3B...

What does that mean?

With an RTX 3060, you can basically run it at full power and exploit it endlessly.

Overseas developers are all delighted too.

This thing has something.

So, how exactly does a 1.3B document parsing model achieve SOTA accuracy?

But to be honest, in the field of document parsing, annotated data is simply a bottomless pit.

Layouts are bizarre and varied, professional formulas, code, and charts are intricate, and relying purely on manual annotation is not only astronomically expensive, but the human error rate itself is terrifyingly high.

You simply cannot exhaust the long-tail bad cases in the real world. Engineers painstakingly pick out bad samples, manually correct them, retrain, and finally release a new version—only for it to still fail when encountering new bizarre layouts.

The speed of data preparation has firmly choked the model's evolution.

The team made a rather radical exploration in this regard.

They realized that document parsing has long had an easily overlooked characteristic:

Every input has an objectively determined ground truth answer.

Whether the text is correct, whether tables have errors across rows or columns, whether formula symbols are missing—all of it can be precisely measured.

Since correctness can be objectively judged, why wait for humans to clean up the mess?

After this thought experiment, OXR made a decision:

In the post-training stage, introduce a recursive self-improvement (RSI) training paradigm for document parsing.

This RSI process can be roughly divided into three stages:

1. Prediction & Classification

OXR first parses massive amounts of unlabeled real documents.

Next, a General VLM steps in, combining the original image, parsing results, and evaluation criteria to divide samples into three categories:

Keeping the Uncertain category is a rather counterintuitive piece of know-how.

If you give the model only two black-and-white options, Correct and Wrong, the judging model's hesitation will be forcibly packaged as a definite label, ultimately feeding poison into the next round of training.

With Uncertain, samples that can't be determined have a fallback.

In addition to the VLM, OXR also performs multiple rollout samplings on itself.

If a document produces wildly different results across several runs, it means the model lacks confidence; even if it happens to guess correctly once, it will be directly thrown into the Uncertain set.

Finally, under the premise that the semantics remain completely unchanged, the system applies slight image perturbations to these high-disagreement samples (small scaling, compression, slight blurring, or translation).

If the disagreement further expands after adding perturbations, it indicates that the model is overly sensitive to this type of harmless visual change, and this perturbation action will be immediately recorded as a core cause, directly guiding the next round of targeted reinforcement.

2. Attribution & Training Planning

After filtering out the wrong samples and weak samples, you absolutely must not just feed them randomly to the model; you need to figure out exactly where it fails.

At this stage, the General VLM acts like an experienced doctor, diagnosing each sample one by one: is it a failure in cross-column cell alignment, or an error in recognizing cursive handwriting?

The diagnosed natural-language causes are fed into SigLIP2 to extract feature vectors, and then clustered through an independent clustering model to form clear reason clusters.

For example, a financial statement and a chemistry lab record table may have nothing in common in appearance or text, but if attribution clustering finds that they both fail on some rare cross-row, cross-column structure, this will be distilled into a common data construction direction.

OXR's specific approach is: reason clusters provide the structural difficulties, and the real sample pool provides references for layout and visual style.

Each time new data is generated, layout styles are grabbed from real samples as anchors, and the structural difficulties that need to be overcome are naturally integrated into them.

The visual weak points previously discovered in perturbation testing are also systematically incorporated into the synthesis pipeline, forcing the model to repeatedly desensitize itself through practice.

Finally, the generated data undergoes strict specification validation and flows directly back to train the next generation of OXR.

After one round is completed, the model's shortcomings are addressed; when a new round of real documents is run, new shortcomings will be exposed, driving the next round of iteration.

RSI has thus taken shape.

The data distribution becomes increasingly complete and solid with each round of iteration.

This is also why OXR can achieve such a high intelligence density.

When data quality is high enough, the model no longer needs to rely on bloated parameter counts for rote memorization.

On a single RTX PRO 6000, OXR can parse 100 pages of documents end to end in only 101.0 seconds, ranking first among open-source solutions.

At first I thought it was because of the small parameter count.

But later I realized that in document parsing, being small alone cannot achieve this.

There is a lot of counterintuitive know-how behind this.

Looking back at the technological evolution of document parsing over the past two years, the industry has actually gone through several stages.

At first, everyone built pipelines, using a separate proprietary model for detection, recognition, and ordering, which was extremely bloated when assembled and had sky-high maintenance costs.

Later, VLMs became popular, and everyone realized that all tasks are essentially image-to-sequence mapping, so they began trying to use end-to-end large models to handle everything at once, letting the large model directly output text and layout for the entire page.

Although elegant, this approach is slow as a snail, because autoregressive generation of long sequences is too compute-intensive.

Later, PaddleOCR-VL took a very elegant step:

It merged layout analysis (LA) and reading order (RO) into a small model, followed by a recognition model, creating two models and two stages.

This brought layout detection speed back to the millisecond level of traditional object detection and directly became the industry-standard solution for the past two years.

But is that perfect?

No.

For every additional layout annotation, a reading-order annotation must be forcibly attached, directly doubling the difficulty of data processing.

After in-depth reasoning, the OXR team proposed a completely different task-level restructuring solution:

An independent lightweight model handles layout analysis (LA), while the main OXR model simultaneously handles content recognition and reading order determination (RO).

That is:

Layout Analysis -> Recognition + Reading Order

This seemingly simple split-and-combination brings two huge qualitative changes in engineering:

First, it completely eliminates the performance loss caused by reading order.

In the entire pipeline, the time required to determine reading order was originally far less than that for text recognition.

After OXR merges reading-order determination into the main model, not only does it not add extra time, but because the main model itself has semantic understanding capability (Content-Aware), the accuracy of reading-order determination is greatly improved.

At the same time, because the preceding layout analysis model (LA) has shed the complex ordering burden, its task becomes extremely pure, allowing it to directly adopt a lighter and more efficient detection model, and overall throughput takes off immediately.

Second, the layout analysis module is independent and fully pluggable.

Different businesses have completely conflicting layout requirements, which is a long-tail problem:

For an area with mixed images and text, the content team wants it extracted as a whole image, while the knowledge base team wants the text inside extracted as body text;

For a densely arranged four-panel comic, business A wants it cropped as a whole, while business B wants it split into four small images;

...

In short, it is very hard to define and impossible to satisfy everyone.

But in OXR's architecture, layout analysis is a completely independent, pluggable module—

achieving maximum freedom of business adaptation with minimal modification cost.

If your business has extremely special layout partitioning requirements, such as processing ancient books or medical bills, you can completely replace it seamlessly with your own trained private layout analysis model,

and the backend can continue to freely leverage OXR's powerful content recognition capabilities.

Of course, since it was born for the developer community, no matter how extravagantly a model is hyped, if it can't be deployed, it's all for nothing.

Yes, completely free, pure free-riding...

The official team says that, at least as long as the server load remains sustainable, they will keep maintaining the open-source free service.

There's no reason not to jump on this and ride it hard!

So I hurriedly integrated this API into my own writing app.

Honestly, for PDF parsing, compared with the previous solution, the results are far, far better.

By the way, for those who want to deploy locally, that's no problem either. OXR is fully adapted to mainstream inference frameworks.

You don't need to painstakingly recompile vLLM yourself; you can spin up the service with one click by mounting the plugin in the official Docker or CLI.

Currently, OXR's code, weights, and the entire reproduction pipeline for the benchmark evaluation have all been made public.

The link is at the end of the article; those interested can grab it themselves.

Of course, if you have higher requirements, such as handling more formats like Word, PPT, Excel, or pursuing more complete post-processing customization capabilities, you can also look into the officially maintained commercial-grade engine.

Why, today, do we still need to expend so much effort grinding away at document parsing?

On a day when Agents are fully taking over workflows across all industries, the most core and most professional private-domain knowledge accumulated by humanity over the past decades is, for the vast majority, still lying quietly inside those PDFs, scans, and reports with chaotic formats and complex layouts.

No matter how smart a large language model is, if the input fed to it is a pile of character garbage with scrambled order and broken tables, all it can spit out is hallucinations.

In the past, many people treated OCR as low-tech grunt work, but today, from OCR to OXR, the mission it undertakes has completely changed.

It is the bridge for human-computer interaction, reconstructing all the complex multimodal documents in the human world, losslessly and completely, into structured assets that are extremely friendly to Agents.

Honestly, seeing a domestic team doing this kind of work close to engineering deployment is actually quite exciting.

Too many people have their eyes fixed on fancy narratives, chasing cutting-edge technology every day, which instead makes us forget one very important thing—

What do users actually need?

Now that LLMs have developed to this stage, the answer to this question is definitely not higher model intelligence.

The bigger bottleneck lies in those fields that no one has yet pioneered and that require getting hands dirty.

If a consensus can be formed on this—

the infrastructure belonging to Agents will, at a visible speed, fall into place piece by piece like a jigsaw puzzle.