让大模型在回答时像推理一样分析信息冲突，提升可信度。
From Facts to Conclusions : Integrating Deductive Reasoning in Retrieval-Augmented LLMs
- 分三阶段结构化推理：裁决文档、分析矛盾、融合结论。
- 使用评判模型评估答案正确性与拒答合理性，提升可信度。
- 适合需要高可靠性的问答系统，如医疗、法律领域。
检索增强生成（RAG）将大语言模型（LLM）与外部证据结合，但在来源冲突、过时或主观时表现不佳。现有方法各自解决单一问题，缺乏统一推理监督。本文提出一种推理轨迹增强的RAG框架，包含三个阶段：（1）文档级裁决，（2）冲突分析，（3）基于证据的合成，生成带引用的答案或合理拒答。引入冲突感知信任评分（CATS）管道，利用大模型作为评判者，评估答案的依据性、事实正确性、拒答准确性及行为一致性。构建了包含539个查询的推理数据集与评估流程，为可解释的冲突感知RAG系统奠定基础。实验表明显著优于基线，尤其在Qwen上，端到端答案正确率从0.069提升至0.883，行为一致性从0.074升至0.722。
原文摘要 · Abstract (English)
Retrieval-Augmented Generation (RAG) grounds large language models (LLMs) in external evidence, but fails when retrieved sources conflict or contain outdated or subjective information. Prior work address these issues independently but lack unified reasoning supervision. We propose a reasoning-trace-augmented RAG framework that adds structured, interpretable reasoning across three stages : (1) document-level adjudication, (2) conflict analysis, and (3) grounded synthesis, producing citation-linked answers or justified refusals. A Conflict-Aware Trust-Score (CATS) pipeline is introduced which evaluates groundedness, factual correctness, refusal accuracy, and conflict-behavior alignment using an LLM-as-a-Judge. Our 539-query reasoning dataset and evaluation pipeline establish a foundation for conflict-aware, interpretable RAG systems. Experimental results demonstrate substantial gains over baselines, most notably with Qwen, where Supervised Fine-Tuning improved End-to-End answer correctness from 0.069 to 0.883 and behavioral adherence from 0.074 to 0.722.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。