让大模型在知识图谱推理时学会自知，避免盲目自信。
When to Trust: A Causality-Aware Calibration Framework for Accurate Knowledge Graph Retrieval-Augmented Generation
- 用反事实提示暴露检索结果的不确定性，识别不可靠信息
- 通过多轮重评分机制稳定预测，使置信度更贴近真实准确率
- 适合对可靠性要求高的医疗、金融等高风险领域应用
知识图谱增强生成（KG-RAG）通过引入结构化知识，使大语言模型在复杂任务中实现更精准、可解释的推理。然而现有KG-RAG模型普遍存在过度自信问题，在检索子图不完整或不可靠时仍给出高置信度答案，影响其在高风险场景的应用。为此，我们提出Ca2KG——一种因果感知的校准框架。该框架结合反事实提示以揭示知识质量与推理可靠性中的检索依赖性不确定性，并采用基于面板的重评分机制，稳定不同干预下的预测结果。在两个复杂问答数据集上的大量实验表明，Ca2KG在保持甚至提升预测准确率的同时，显著改善了模型校准性能。
原文摘要 · Abstract (English)
Knowledge Graph Retrieval-Augmented Generation (KG-RAG) extends the RAG paradigm by incorporating structured knowledge from knowledge graphs, enabling Large Language Models (LLMs) to perform more precise and explainable reasoning. While KG-RAG improves factual accuracy in complex tasks, existing KG-RAG models are often severely overconfident, producing high-confidence predictions even when retrieved sub-graphs are incomplete or unreliable, which raises concerns for deployment in high-stakes domains. To address this issue, we propose Ca2KG, a Causality-aware Calibration framework for KG-RAG. Ca2KG integrates counterfactual prompting, which exposes retrieval-dependent uncertainties in knowledge quality and reasoning reliability, with a panel-based re-scoring mechanism that stabilises predictions across interventions. Extensive experiments on two complex QA datasets demonstrate that Ca2KG consistently improves calibration while maintaining or even enhancing predictive accuracy.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。