arXiv:2510.25867cs.LG2025-10中稿 · ICLR被引 2

用开源医学文献自动生成高质量医疗视觉问答数据,提升模型表现。

Synthesizing High-Quality Visual Question Answering from Medical Documents with Generator-Verifier LMMs

  • 基于图文和文本引用,生成结构化多选题
  • 13,087个问题覆盖14,803张图像,跨13种成像模态
  • 验证机制确保数据质量,适合训练可信医疗AI

大型多模态模型在联合推理图像与文本回答医学问题方面能力日益增强,但缺乏大规模、公开可用的高质量语料库阻碍了通用医疗视觉问答(VQA)系统的训练。本文提出MedVLSynther,一种基于评分标准的生成-验证框架,直接从开放生物医学文献中生成高质量的多选题,以图表、图注及文中引用为条件。生成器在可机器校验的JSON模式下生成自包含题干与互斥选项;多阶段验证器执行自洽性、唯一正确答案、临床有效性、图文一致性等关键检查,给予细粒度正向评分并惩罚常见错误后才接受。该流程应用于PubMed Central,构建出MedSynVQA数据集:包含13,087个经审核的问题,覆盖14,803张图像,涉及13种成像模态与28个解剖区域。使用可验证奖励通过强化学习训练开放权重的LMM,在六个医疗VQA基准上提升准确率,3B模型平均达55.85,7B模型达58.15,最高达77.57(VQA-RAD)和67.76(PathVQA),优于现有强医疗LMM。消融实验表明生成与验证均必要,更多经验证数据持续提升性能;针对性污染分析未发现评估集泄漏。该方法完全基于开放文献与开放权重模型,提供可审计、可复现、隐私安全的规模化医疗VQA数据生成路径。

原文摘要 · Abstract (English)

Large Multimodal Models (LMMs) are increasingly capable of answering medical questions that require joint reasoning over images and text, yet training general medical VQA systems is impeded by the lack of large, openly usable, high-quality corpora. We present MedVLSynther, a rubric-guided generator-verifier framework that synthesizes high-quality multiple-choice VQA items directly from open biomedical literature by conditioning on figures, captions, and in-text references. The generator produces self-contained stems and parallel, mutually exclusive options under a machine-checkable JSON schema; a multi-stage verifier enforces essential gates (self-containment, single correct answer, clinical validity, image-text consistency), awards fine-grained positive points, and penalizes common failure modes before acceptance. Applying this pipeline to PubMed Central yields MedSynVQA: 13,087 audited questions over 14,803 images spanning 13 imaging modalities and 28 anatomical regions. Training open-weight LMMs with reinforcement learning using verifiable rewards improves accuracy across six medical VQA benchmarks, achieving averages of 55.85 (3B) and 58.15 (7B), with up to 77.57 on VQA-RAD and 67.76 on PathVQA, outperforming strong medical LMMs. A Ablations verify that both generation and verification are necessary and that more verified data consistently helps, and a targeted contamination analysis detects no leakage from evaluation suites. By operating entirely on open literature and open-weight models, MedVLSynther offers an auditable, reproducible, and privacy-preserving path to scalable medical VQA training data.

医疗VQA数据生成多模态模型验证机制

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。