用贝叶斯网络追踪多跳问答中各环节的不确定性,提升AI系统的可信度。
Bayesian Uncertainty Propagation for Agentic RAG Pipelines: A Proof-of-Concept Study on Multi-Hop Question Answering

- 通过语义差异和生成自评生成各阶段不确定性信号
- 在HotpotQA上贝叶斯传播显著提升失败点识别效果
- 适合关注AI决策可信性与故障预警的研究者
可信部署智能体增强型检索生成(Agentic RAG)系统需要估计多阶段推理流程可能出错的时机。本文提出一种不确定性感知的Agentic RAG框架,规划、评估与生成阶段分别基于语义分歧与生成器自评产生不确定性信号,并通过贝叶斯网络(BN)传播,以估计系统级不确定性并提供流程中各节点的潜在故障指示。在StrategyQA与HotpotQA数据集上使用GPT-3.5-Turbo与GPT-4.1-Nano进行评估,采用受试者工作特征曲线下面积(AUROC)、准确率-拒绝曲线下面积(AUARC)、期望校准误差(ECE)与Brier Score衡量区分度、选择性预测与校准性。结果表明,贝叶斯传播在需跨阶段推理的HotpotQA上表现更优,而StrategyQA暴露了校准偏差与上游信号不可靠的问题。该研究将贝叶斯不确定性传播定位为监控Agentic RAG系统的有前景但初步的方法,未来需在离岸风电(OSW)运维决策支持等工业场景中进一步验证。
原文摘要 · Abstract (English)
Trustworthy deployment of Agentic Retrieval-Augmented Generation (RAG) systems requires mechanisms for estimating when multi-stage reasoning pipelines may fail. This paper presents an uncertainty-aware Agentic Retrieval-Augmented Generation (RAG) framework in which planner, evaluator and generator stages produce uncertainty signals derived from semantic divergence and generator self-evaluation. These signals are propagated through a Bayesian Network (BN) to estimate system-level uncertainty and provide node-level indicators of potential failure points across the workflow. The approach is evaluated on StrategyQA and HotpotQA using GPT-3.5-Turbo and GPT-4.1-Nano, with Area Under the Receiver Operating Characteristic Curve (AUROC), Area Under the Accuracy-Rejection Curve (AUARC), Expected Calibration Error (ECE), and Brier Score used to assess discrimination, selective prediction and calibration. Results show that Bayesian propagation is more effective on HotpotQA, where uncertainty accumulates across multi-hop reasoning stages, while StrategyQA exposes limitations caused by miscalibration and unreliable upstream signals. The study positions Bayesian uncertainty propagation as a promising but preliminary mechanism for monitoring Agentic RAG systems, with future validation required in industrial domains such as Offshore Wind (OSW) maintenance decision support.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。