arXiv:2601.00513cs.LG2026-01中稿 · AAAI被引 2

小模型答对但理由错,需用新方法验证推理过程。

When Small Models Are Right for Wrong Reasons: Process Verification for Trustworthy Agents

  • 提出基于推理路径的可信度评分RIS,量化推理质量。
  • 50%-69%正确答案源于错误推理,传统准确率无法发现此问题。
  • 小模型用检索增强可提升推理可靠性,自我反思反而有害。

将参数量为7-9B的小语言模型部署为自主智能体,需要对其推理过程建立信任,而不仅仅是输出结果。我们揭示了一个关键的可靠性危机:在这些模型中,50%-69%的正确答案实际上基于根本性错误的推理——即“因错误原因而正确”现象,这在标准准确率指标下完全不可见。通过对三种模型在10,734条推理轨迹上的分析,我们提出了推理完整性评分(RIS),该指标具有较高的评判者间一致性(κ=0.657)。研究挑战了现有实践:尽管检索增强生成(RAG)显著提升了推理完整性(Cohen's d=0.23–0.93),但元认知干预如自我批判在所评估任务中往往损害性能(d=-0.14至-0.33)。机制分析显示,RAG通过引入外部证据来锚定计算,使错误率降低7.6%,而元认知则在模型容量不足时加剧混乱。为支持实际部署,我们设计了一个神经分类器,可将验证能力压缩并实现100倍加速,达到0.86的F1分数。结果表明,对于可信智能体而言,仅靠准确率是危险的;必须采用基于过程的验证机制。

原文摘要 · Abstract (English)

Deploying small language models (7-9B parameters) as autonomous agents requires trust in their reasoning, not just their outputs. We reveal a critical reliability crisis: 50-69\% of correct answers from these models contain fundamentally flawed reasoning -- a ``Right-for-Wrong-Reasons'' phenomenon invisible to standard accuracy metrics. Through analysis of 10,734 reasoning traces across three models and diverse tasks, we introduce the Reasoning Integrity Score (RIS), a process-based metric validated with substantial inter-rater agreement ($κ=0.657$). Conventional practices are challenged by our findings: while retrieval-augmented generation (RAG) significantly improves reasoning integrity (Cohen's $d=0.23$--$0.93$), meta-cognitive interventions like self-critique often harm performance ($d=-0.14$ to $-0.33$) in small models on the evaluated tasks. Mechanistic analysis reveals RAG succeeds by grounding calculations in external evidence, reducing errors by 7.6\%, while meta-cognition amplifies confusion without sufficient model capacity. To enable deployment, verification capabilities are distilled into a neural classifier achieving 0.86 F1-score with 100$\times$ speedup. These results underscore the necessity of process-based verification for trustworthy agents: accuracy alone is dangerously insufficient when models can be right for entirely wrong reasons.

小模型推理验证可信智能体RAG

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。