用合成数据训练检索器,让AI更准理解扫描版专业文档。
Docs2Synth: A Synthetic Data Trained Retriever Framework for Scanned Visually Rich Documents Understanding
- 通过智能代理自动生成并验证多样化的问答对
- 在多个基准上显著提升领域泛化能力,无需人工标注
- 适合医疗、金融等敏感低资源领域快速部署
受监管领域的视觉丰富文档理解(VRDU)极具挑战性,因扫描文档常含敏感、动态且领域特异的知识。这导致两大难题:模型适配缺乏人工标注,预训练模型难以跟上领域事实更新。尽管多模态大语言模型(MLLM)具备强零样本能力,仍存在幻觉和领域定位不足问题。相反,判别式视觉-语言预训练模型(VLPM)虽具可靠定位能力,但需昂贵标注覆盖新领域。我们提出Docs2Synth,一种合成监督框架,支持私有与低资源领域的检索引导推理。该框架自动处理原始文档集合,通过基于智能体的系统生成并验证多样化QA对,并训练轻量级视觉检索器以提取领域相关证据。推理时,检索器与MLLM通过迭代检索-生成循环协作,降低幻觉并提升响应一致性。我们进一步将Docs2Synth封装为易用Python包,支持跨多种真实场景的即插即用部署。在多个VRDU基准上的实验表明,Docs2Synth在无需人工标注的前提下显著增强领域定位与泛化能力。
原文摘要 · Abstract (English)
Document understanding (VRDU) in regulated domains is particularly challenging, since scanned documents often contain sensitive, evolving, and domain specific knowledge. This leads to two major challenges: the lack of manual annotations for model adaptation and the difficulty for pretrained models to stay up-to-date with domain-specific facts. While Multimodal Large Language Models (MLLMs) show strong zero-shot abilities, they still suffer from hallucination and limited domain grounding. In contrast, discriminative Vision-Language Pre-trained Models (VLPMs) provide reliable grounding but require costly annotations to cover new domains. We introduce Docs2Synth, a synthetic-supervision framework that enables retrieval-guided inference for private and low-resource domains. Docs2Synth automatically processes raw document collections, generates and verifies diverse QA pairs via an agent-based system, and trains a lightweight visual retriever to extract domain-relevant evidence. During inference, the retriever collaborates with an MLLM through an iterative retrieval--generation loop, reducing hallucination and improving response consistency. We further deliver Docs2Synth as an easy-to-use Python package, enabling plug-and-play deployment across diverse real-world scenarios. Experiments on multiple VRDU benchmarks show that Docs2Synth substantially enhances grounding and domain generalization without requiring human annotations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。