arXiv:2605.27931cs.AI2026-05

用草图生成学术图表,通过检索相似图提升质量与速度。

DiagramRAG: A Lightweight Framework to Retrieve Scientific Diagram for Figure Generation

论文配图:DiagramRAG: A Lightweight Framework to Retrieve Scientific Diagram for Figure Generation
图 1 · 摘自论文原文
  • 将草图和图表转为知识图谱,实现语义与结构双重匹配检索
  • 在DiagramBank和FigureBench上分别达0.848和0.802的F1分数
  • 推理延迟仅35.48秒/样本,适合科研人员快速出图

科学图表对传达复杂方法至关重要。研究人员常通过草图表达早期语义与拓扑意图,但草图通常不完整,难以直接生成高质量图表。现有草图生成方法多重建草图本身,而文本驱动框架未充分利用草图中的结构信息。本文提出DiagramRAG,一种轻量级检索增强框架,基于用户草图检索语义相关且拓扑兼容的参考图表,并用于指导下游生成。为实现高效结构感知检索,将图表表示为知识图谱,构建不同简化层级的草图变体,并训练嵌入模型在共享空间中对齐草图与匹配图表。检索到的参考图提供内容、拓扑与视觉先验,以完成并渲染最终图表。实验表明,DiagramRAG在DiagramBank和FigureBench上的F1分数分别为0.848和0.802,生成质量最优(VLM-as-a-Judge得分为7.170),推理延迟降至每样本35.48秒。代码与数据已公开。

原文摘要 · Abstract (English)

Scientific diagrams are essential for communicating complex methodologies in academic papers. A natural way for researchers to specify such diagrams is through rough sketches, where text labels, connectors, and spatial arrangements express early semantic and topological intentions. However, sketches are usually incomplete, making them insufficient for directly producing publication-quality diagrams. Existing sketch-based generation methods mainly reconstruct the sketch itself, while recent text-driven diagram generation frameworks rely on textual semantics and do not fully exploit the topological structure contained in sketches. In this paper, we introduce DiagramRAG, a lightweight retrieval-augmented framework for sketch-based scientific diagram completion. Given a user sketch, DiagramRAG retrieves reference diagrams that are both semantically relevant to the sketch content and topologically compatible with its structure, and uses them to guide downstream diagram generation. To enable efficient structure-aware retrieval, we represent diagrams as knowledge graphs, synthesize sketch variants at different simplification levels, and train an embedding model to align sketches with compatible diagrams in a shared space. The retrieved references further provide content, topology, and visual priors for completing and rendering the final diagram. Experiments show that DiagramRAG achieves F1-scores of 0.848 and 0.802 on DiagramBank and FigureBench, respectively, and improves generation quality with the best VLM-as-a-Judge score of 7.170, while reducing inference latency to 35.48 seconds per sample. Our code and data are available at https://anonymous.4open.science/r/DiagramRAG-A262 and https://huggingface.co/datasets/anonymous-review-a262/DiagramSketch.

图表生成草图理解检索增强知识图谱

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。