用异构锚点提示实现文档图像高效解析，提升布局与内容生成质量。
Dolphin: Document Image Parsing via Heterogeneous Anchor Prompting
- 先按阅读顺序生成布局元素，再用任务提示并行解析内容
- 在3000万样本上训练，在多基准上达到最先进性能
- 轻量架构支持并行处理，适合高效率文档自动化场景
文档图像解析因文本段落、图表、公式、表格等元素复杂交织而极具挑战。现有方法或依赖多个专用模型，或直接自回归生成整页内容，存在集成开销大、效率低、版式结构退化等问题。为此，我们提出Dolphin（文档图像解析通过异构锚点提示），采用先分析后解析的范式：第一阶段生成按阅读顺序排列的布局元素序列；第二阶段将这些异构元素作为锚点，结合任务特定提示，实现并行内容解析。为训练Dolphin，我们构建了超过3000万样本的大规模数据集，涵盖多粒度解析任务。在主流及自建基准上的综合评估表明，Dolphin在页面级与元素级任务中均达到最先进水平，同时凭借轻量架构和并行解析机制实现优异效率。代码与预训练模型已公开于https://github.com/ByteDance/Dolphin。
原文摘要 · Abstract (English)
Document image parsing is challenging due to its complexly intertwined elements such as text paragraphs, figures, formulas, and tables. Current approaches either assemble specialized expert models or directly generate page-level content autoregressively, facing integration overhead, efficiency bottlenecks, and layout structure degradation despite their decent performance. To address these limitations, we present \textit{Dolphin} (\textit{\textbf{Do}cument Image \textbf{P}arsing via \textbf{H}eterogeneous Anchor Prompt\textbf{in}g}), a novel multimodal document image parsing model following an analyze-then-parse paradigm. In the first stage, Dolphin generates a sequence of layout elements in reading order. These heterogeneous elements, serving as anchors and coupled with task-specific prompts, are fed back to Dolphin for parallel content parsing in the second stage. To train Dolphin, we construct a large-scale dataset of over 30 million samples, covering multi-granularity parsing tasks. Through comprehensive evaluations on both prevalent benchmarks and self-constructed ones, Dolphin achieves state-of-the-art performance across diverse page-level and element-level settings, while ensuring superior efficiency through its lightweight architecture and parallel parsing mechanism. The code and pre-trained models are publicly available at https://github.com/ByteDance/Dolphin
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。