arXiv:2507.22938cs.CLcs.AI2025-07中稿 · KDD被引 4

用图结构解析通信文档中的流程图,提升多模态问答准确率

A Graph-based Approach for Multi-Modal Question Answering from Flowcharts in Telecom Documents

论文配图:A Graph-based Approach for Multi-Modal Question Answering from Flowcharts in Telecom Documents
图 1 · 摘自论文原文
  • 将流程图转为图表示,融合进文本检索增强生成系统
  • 图表示与真实答案的编辑距离更低,检索性能优于纯文本方法
  • 部署时无需推理阶段使用视觉大模型,降低成本

从技术文档中进行问答常需理解图表(如流程图),纯文本检索增强生成(RAG)系统可能失效。本文利用视觉大语言模型(VLM)生成流程图的图表示,并融入文本RAG系统,实现通信领域多模态问答。提出端到端流程:处理文档、图像类型分类、构建图表示,并集成至文本嵌入管道以实现高效检索。在基于专有通信产品文档构建的QA数据集上评估,经微调的VLM生成的图表示与真实标注的编辑距离更小,表明其对流程图的鲁棒性。此外,该方法使用文本嵌入模型(包括通信领域适配版)即可实现良好检索效果,且推理阶段无需部署VLM,显著降低系统成本。

原文摘要 · Abstract (English)

Question-Answering (QA) from technical documents often involves questions whose answers are present in figures, such as flowcharts or flow diagrams. Text-based Retrieval Augmented Generation (RAG) systems may fail to answer such questions. We leverage graph representations of flowcharts obtained from Visual large Language Models (VLMs) and incorporate them in a text-based RAG system to show that this approach can enable image retrieval for QA in the telecom domain. We present the end-to-end approach from processing technical documents, classifying image types, building graph representations, and incorporating them with the text embedding pipeline for efficient retrieval. We benchmark the same on a QA dataset created based on proprietary telecom product information documents. Results show that the graph representations obtained using a fine-tuned VLM model have lower edit distance with respect to the ground truth, which illustrate the robustness of these representations for flowchart images. Further, the approach for QA using these representations gives good retrieval performance using text-based embedding models, including a telecom-domain adapted one. Our approach also alleviates the need for a VLM in inference, which is an important cost benefit for deployed QA systems.

多模态问答流程图理解图表示通信文档

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。