从教材中自动提取8.3万组带图问答数据，成本降50倍且结构准确。
FlipVQA: Scaling Multi-modal Instruction Tuning via Textbook-to-Knowledge Synthesis
- 通过解析跨页复杂排版，恢复问题-答案-图像的完整关联
- 构建83,000对多模态问答数据，结构准确率超96%
- 适合需要高质量推理训练数据的研究者使用
教材是人类验证过的推理知识宝库，但其多栏排版、跨页问答分离和图文交错等复杂布局，使得自动化提取结构化问答对极为困难。现有方法或从零合成数据，缺乏真实问题背景；或依赖昂贵的人工标注，难以扩展。我们提出FlipVQA-Miner自动化流水线，解决OCR解析文档中的长距离逻辑依赖与跨页断续问题，即使答案位于配套分册也能恢复完整的问答图关联。后续多阶段精修流程将原始提取结果转化为可直接用于AI训练的监督信号。基于该方法，我们构建了包含83,000组问答与视觉问答对的FlipVQA-83K数据集，覆盖11个学科领域，在结构保真度（F₁ > 0.96）的前提下实现人工标注50倍的成本节省。在该数据集上微调的模型展现出显著提升的推理能力与跨领域泛化性能，确立了一种基于人类知识的数据构建可扩展范式。数据集及完整生成与精修方法详见https://github.com/OpenDCAI/DataFlow-VQA。
原文摘要 · Abstract (English)
Textbooks are among the richest repositories of human-verified reasoning knowledge, yet their complex layouts contain multi-column typesetting, cross-page question answer separation, and interleaved figures, make automated extraction of structured QA and VQA pairs extremely challenging. Existing alternatives either synthesize data from scratch, which lacks authentic problem contexts, or rely on costly expert annotation that cannot scale. We propose $\textbf{FlipVQA-Miner}$, an automated pipeline that resolves long-range logical dependencies and cross-page discontinuities in OCR-parsed documents, recovering coherent question--answer--figure associations even when answers reside in separate companion volumes. A subsequent multi-stage curation pipeline transforms these raw extractions into AI-ready supervision signals. Using FlipVQA-Miner, we construct $\textbf{FlipVQA-83K}$, comprising 83K QA and VQA pairs spanning 11 academic disciplines, at a $\textbf{50$\times$}$ cost saving compared to manual annotation while maintaining high structural fidelity ($F_1 > 0.96$). Models fine-tuned on FlipVQA-83K demonstrate significantly improved reasoning ability and cross-domain generalization, establishing a scalable paradigm for human-knowledge-grounded data curation. Our dataset and the complete data generating and curating methods can be found in https://github.com/OpenDCAI/DataFlow-VQA .
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。