arXiv:2506.01344cs.CL2025-06EMNLP被引 6

用符号化智能体精准追踪流程图决策路径,解决大模型幻觉问题。

Follow the Flow: Fine-grained Flowchart Attribution with Neurosymbolic Agents

  • 构建符号化图谱,动态交互推理实现细粒度归因
  • 在多风格流程图上比基线提升10-14%准确率
  • 适合需要高可信解释的医疗、工程等关键领域

流程图是可视化决策过程的关键工具,但其非线性结构和复杂的图文关系使大语言模型(LLM)难以准确解析,视觉语言模型常会虚构不存在的连接与路径,导致物流、医疗、工程等关键领域自动化处理可靠性下降。本文提出细粒度流程图归因任务,旨在将LLM的回答与流程图具体组件关联,确保预测可验证并增强可解释性。为此,我们提出FlowPathAgent,一种基于图推理的神经符号智能体:先对流程图进行分割,转化为结构化符号图,再通过代理式动态交互生成归因路径。此外,我们构建了FlowExplainBench,一个涵盖多种风格、领域和问题类型的新型评估基准。实验表明,FlowPathAgent在流程图问答中有效抑制视觉幻觉,相比强基线在所提数据集上提升10-14%。

原文摘要 · Abstract (English)

Flowcharts are a critical tool for visualizing decision-making processes. However, their non-linear structure and complex visual-textual relationships make it challenging to interpret them using LLMs, as vision-language models frequently hallucinate nonexistent connections and decision paths when analyzing these diagrams. This leads to compromised reliability for automated flowchart processing in critical domains such as logistics, health, and engineering. We introduce the task of Fine-grained Flowchart Attribution, which traces specific components grounding a flowchart referring LLM response. Flowchart Attribution ensures the verifiability of LLM predictions and improves explainability by linking generated responses to the flowchart's structure. We propose FlowPathAgent, a neurosymbolic agent that performs fine-grained post hoc attribution through graph-based reasoning. It first segments the flowchart, then converts it into a structured symbolic graph, and then employs an agentic approach to dynamically interact with the graph, to generate attribution paths. Additionally, we present FlowExplainBench, a novel benchmark for evaluating flowchart attributions across diverse styles, domains, and question types. Experimental results show that FlowPathAgent mitigates visual hallucinations in LLM answers over flowchart QA, outperforming strong baselines by 10-14% on our proposed FlowExplainBench dataset.

流程图分析可解释AI神经符号

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。