让大模型推理更可信，只基于有依据的假设
Audit-of-Understanding: Posterior-Constrained Inference for Mathematical Reasoning in Language Models
- 分三步审计推理前提，只允许经验证的前提参与推导
- 在GSM8K等数据集上准确率提升最高达45%
- 适合需要高可靠性推理的应用场景
大语言模型常生成看似合理但依赖未经证实假设的推理过程，导致幻觉结论。现有方法多聚焦事实性幻觉或依赖事后验证，对推理引发的幻觉缺乏有效应对。本文提出审计理解（AoU）框架，通过三个阶段约束推理：(1) 将问题分解为候选假设，(2) 审计其支持度，(3) 仅基于已验证前提进行推导。形式上，AoU是后验约束推理，与选择性预测和拒绝学习相关。贡献包括：(i) 在完美验证下的理论保证，(ii) 不完美审计下的超额风险界，(iii) 可行性分析。实验表明，AoU在GSM8K、MultiArith和SVAMP上均提升准确率与可信度，相比CoT、Self-Consistency和CoT-Decoding，在GSM8K上最高提升30%，MultiArith上达45%，SVAMP上稳定提升20%–28%。代码已公开。
原文摘要 · Abstract (English)
Large language models (LLMs) often generate reasoning traces that appear coherent but rest on unsupported assumptions, leading to hallucinated conclusions. Prior work mainly addresses factual hallucinations or relies on post-hoc verification, leaving reasoning-induced hallucinations largely unaddressed. We propose Audit-of-Understanding (AoU), a framework that constrains inference to validated premises through three phases: (1) decomposing a query into candidate assumptions, (2) auditing their support, and (3) conditioning inference only on the validated subset. Formally, AoU is \emph{posterior-constrained inference}, connecting to selective prediction and rejection learning. Our contributions are threefold: (i) theoretical guarantees under perfect validation, (ii) excess-risk bounds under imperfect audits, and (iii) tractability analysis. Empirically, AoU improves both accuracy and faithfulness on GSM8K, MultiArith, and SVAMP, achieving up to +30% gains on GSM8K, +45% on MultiArith, and consistent +20--28% improvements on SVAMP over Chain-of-Thought, Self-Consistency, and CoT-Decoding. Code is available at https://anonymous.4open.science/r/audit-of-understanding-E28B.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。