无需知识蒸馏,用自生成数据+迭代优化提升文档转换模型性能。
POINTS-Reader: Distillation-Free Adaptation of Vision-Language Models for Document Conversion
- 先生成大量合成数据训练基础模型,再用真实文档迭代优化。
- 在多个公开数据集上超越同类模型,尤其在复杂表格和公式上表现优异。
- 适合需要高精度文档转换但缺乏标注数据的场景。
高质量标注数据对训练准确的文档转换模型至关重要,尤其在包含表格、公式和多栏文本等复杂格式的领域。然而,人工标注成本高昂且耗时,而现有模型自动标注又常因准确性不足难以应对挑战。因此,依赖教师模型输出进行知识蒸馏会严重限制学生模型的实际表现。本文提出一种完全自动化的无蒸馏框架,包含两个阶段:第一阶段通过生成大规模多样化合成数据,使模型具备统一格式提取关键元素的初始能力;第二阶段采用自改进策略,将已在合成数据上训练的模型进一步适配真实文档。具体而言,先用微调后的模型标注真实文档,再通过一系列过滤策略验证标注质量,最后在经验证的数据集上重新训练模型。通过反复迭代该过程,持续提升模型转换能力和数据生成质量。我们基于POINTS-1.5模型训练出POINTS-Reader,在多个公开数据集上超越众多同类规模或更大尺寸的公共及私有模型,相关代码已开源。
原文摘要 · Abstract (English)
High-quality labeled data is essential for training accurate document conversion models, particularly in domains with complex formats such as tables, formulas, and multi-column text. However, manual annotation is both costly and time-consuming, while automatic labeling using existing models often lacks accuracy in handling such challenging scenarios. Consequently, training student models by distilling outputs from teacher models can significantly limit their performance in real-world applications. In this paper, we propose a fully automated, distillation-free framework comprising two stages for constructing high-quality document extraction datasets and models capable of handling diverse document formats and layouts. In the first stage, we introduce a method for generating large-scale, diverse synthetic data, which enables a model to extract key elements in a unified format with strong initial performance. In the second stage, we present a self-improvement approach that further adapts the model, initially trained on synthetic data, to real-world documents. Specifically, we first use the fine-tuned model to annotate real documents, then apply a suite of filtering strategies to verify annotation quality, and finally retrain the model on the verified dataset. By iteratively repeating this process, we progressively enhance both the model's conversion capabilities and the quality of the generated data. We train a public POINTS-1.5 model to obtain POINTS-Reader, which surpasses many existing public and proprietary models of comparable or larger size. Our model is available at https://github.com/Tencent/POINTS-Reader.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。