将通用视觉语言模型改造为精准文档解析专家,解决结构幻觉问题。
FireRed-OCR Technical Report
- 基于几何与语义特征构建合成数据集,解决高质量标注数据稀缺问题。
- 采用三阶段渐进式训练,实现从像素感知到逻辑结构生成的跃迁。
- 在多个指标上超越现有模型,适合工业级文档智能处理场景。
我们提出 FireRed-OCR,一个系统性框架,将通用视觉语言模型(VLM)专业化为高性能光学字符识别(OCR)模型。尽管大型视觉语言模型具备强大泛化能力,但在处理复杂文档时常出现“结构幻觉”,限制其在工业OCR中的应用。本文设计的 FireRed-OCR 框架基于 Qwen3-VL 构建,旨在使模型成为像素级精准的文档结构解析专家。针对高质量结构化数据匮乏的问题,我们构建了“几何+语义”数据工厂:不采用传统随机采样,而是通过几何特征聚类与多维标签机制合成并筛选高平衡数据集,有效应对长尾布局和罕见文档类型。此外,提出三阶段渐进式训练策略:(1) 多任务预对齐,建立模型对文档结构的理解;(2) 专用监督微调,标准化全图 Markdown 输出;(3) 格式约束组相对策略优化(GRPO),利用强化学习强制保证语法正确性与结构完整性(如表格闭合、公式语法)。在 OmniDocBench v1.5 上的大量评估显示,FireRed-OCR 总体得分达 92.94%,显著优于 DeepSeek-OCR 2 与 OCRVerse 等强基线,在文本、公式、表格及阅读顺序等维度均表现领先。代码与模型权重已开源,推动“通用 VLM 到专用结构专家”的范式发展。
原文摘要 · Abstract (English)
We present FireRed-OCR, a systematic framework to specialize general VLMs into high-performance OCR models. Large Vision-Language Models (VLMs) have demonstrated impressive general capabilities but frequently suffer from ``structural hallucination'' when processing complex documents, limiting their utility in industrial OCR applications. In this paper, we introduce FireRed-OCR, a novel framework designed to transform general-purpose VLMs (based on Qwen3-VL) into pixel-precise structural document parsing experts. To address the scarcity of high-quality structured data, we construct a ``Geometry + Semantics'' Data Factory. Unlike traditional random sampling, our pipeline leverages geometric feature clustering and multi-dimensional tagging to synthesize and curate a highly balanced dataset, effectively handling long-tail layouts and rare document types. Furthermore, we propose a Three-Stage Progressive Training strategy that guides the model from pixel-level perception to logical structure generation. This curriculum includes: (1) Multi-task Pre-alignment to ground the model's understanding of document structure; (2) Specialized SFT for standardizing full-image Markdown output; and (3) Format-Constrained Group Relative Policy Optimization (GRPO), which utilizes reinforcement learning to enforce strict syntactic validity and structural integrity (e.g., table closure, formula syntax). Extensive evaluations on OmniDocBench v1.5 demonstrate that FireRed-OCR achieves state-of-the-art performance with an overall score of 92.94\%, significantly outperforming strong baselines such as DeepSeek-OCR 2 and OCRVerse across text, formula, table, and reading order metrics. We open-source our code and model weights to facilitate the ``General VLM to Specialized Structural Expert'' paradigm.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。