用知识溯源法检验大模型推理是否靠谱，发现漏洞更精准。
Assessing LLM Reasoning Steps via Principal Knowledge Grounding
- 构建原子知识库，追踪推理每一步的依据来源。
- 通过轻量评估模型，识别知识缺失或误用，准确率显著提升。
- 适合研究模型可解释性、安全验证的开发者与研究人员。
逐步推理已成为大语言模型应对复杂任务的标准方法。然而，这一范式引发了一个根本问题：如何验证模型的推理是否真正基于正确知识？为此，我们提出一个全新的评估套件，系统化地检验中间推理步骤的知识根基。该框架包含三个核心组件：(1) 主要知识收集，构建大规模原子知识库以支撑推理；(2) 基于知识的评估指标，用于衡量模型在推理中对前提知识的回忆与应用程度；(3) 评估用轻量级LLM，专门优化以实现低成本且可靠的指标计算。评估套件在识别知识缺失或误用方面表现出色，为揭示大模型深层推理缺陷提供了关键洞见。此外，我们还展示了这些指标如何融入偏好优化，拓展了知识接地评估的应用场景。
原文摘要 · Abstract (English)
Step-by-step reasoning has become a standard approach for large language models (LLMs) to tackle complex tasks. While this paradigm has proven effective, it raises a fundamental question: How can we verify that an LLM's reasoning is accurately grounded in knowledge? To address this question, we introduce a novel evaluation suite that systematically assesses the knowledge grounding of intermediate reasoning. Our framework comprises three key components. (1) Principal Knowledge Collection, a large-scale repository of atomic knowledge essential for reasoning. Based on the collection, we propose (2) knowledge-grounded evaluation metrics designed to measure how well models recall and apply prerequisite knowledge in reasoning. These metrics are computed by our (3) evaluator LLM, a lightweight model optimized for cost-effective and reliable metric computation. Our evaluation suite demonstrates remarkable effectiveness in identifying missing or misapplied knowledge elements, providing crucial insights for uncovering fundamental reasoning deficiencies in LLMs. Beyond evaluation, we demonstrate how these metrics can be integrated into preference optimization, showcasing further applications of knowledge-grounded evaluation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。