DIAGRAMS自动标注图表问答中的推理依据,提升标注效率。
DIAGRAMS: A Review Framework for Reasoning-Level Attribution in Diagram QA

- 用元模式与数据集适配器解耦界面与格式,支持多类图表数据。
- 模型建议的证据达85.39%精确率和75.30%召回率,接近人工标注。
- 适合需要可解释性标注的视觉推理研究者使用。
图表问答(Diagram QA)需要将每个问题-答案对与推导答案所需的所有视觉区域关联起来,而不仅限于最终答案所在的区域。在图表、流程图、地图、电路图和信息图中构建此类结构化证据耗时费力,且现有标注工具的界面与特定数据集格式紧密耦合。我们提出DIAGRAMS,一个轻量级、基于模式的评审框架,通过内部元模式和数据集适配器,将界面逻辑与数据集特定的JSON结构解耦。给定图像与问题-答案对(含可选候选区域),系统执行条件化证据选择,提出推理所需区域。当缺少问题-答案对或候选区域时,系统可生成并支持人工验证与修正。在六个Diagram QA数据集上,模型建议的证据相对于评审者最终选择,达到85.39%的精确率和75.30%的召回率(微平均)。结果表明,该评审优先框架在减少手动区域创建的同时,保持了与最终推理级归因的高度一致性。我们公开发布演示版本与可安装包,支持数据集审计、基于事实的监督生成与评估。
原文摘要 · Abstract (English)
Diagram question answering (Diagram QA) requires reasoning-level attribution that links each question-answer pair to all visual regions needed to derive the answer, rather than only the region containing the final response. Creating such structured evidence across diagrams, charts, maps, circuits, and infographics is time-consuming, and existing annotation tools tightly couple their interfaces to dataset-specific formats. We present DIAGRAMS, a lightweight, schema-driven review framework that decouples interface logic from dataset-specific JSON structures through an internal meta-schema and dataset adapters. Given an image and QA pair with optional candidate regions, the system performs QA-conditioned evidence selection and proposes the regions required for reasoning. When QA pairs or candidate regions are missing, it generates them and supports human verification and refinement. Across six Diagram QA datasets, model-suggested evidence achieves 85.39% precision and 75.30% recall against reviewer-final selections (micro-averaged). These results indicate that the review-first framework reduces manual region creation while maintaining high agreement with final reasoning-level attributions. We release a public demo and installable package to support dataset auditing, grounded supervision creation, and grounded evaluation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。