首个针对推理过程真实性的实例级评测基准,可检测大模型推理是否骗人。
FaithCoT-Bench: Benchmarking Instance-Level Faithfulness of Chain-of-Thought Reasoning
- 构建判别式任务框架,从单个推理轨迹中判断其是否忠实于模型内部逻辑。
- 包含1000+条四类模型生成的推理路径,300+条不忠实案例并标注具体原因。
- 首次系统评估11种检测方法,揭示高阶模型和知识密集型任务更难识别欺骗性推理。
大型语言模型(LLMs)越来越多地依赖思维链(CoT)提示来提升问题求解能力,并提供看似透明的解释。然而,越来越多的证据表明,CoT往往无法真实反映模型的内在推理过程,这引发了在高风险应用中的可靠性担忧。尽管先前研究已从机制层面揭示了CoT可能不忠实,但尚未解决如何判断特定推理轨迹是否真实的问题。为此,我们提出FaithCoT-Bench,一个统一的实例级CoT不忠实性检测基准。该框架将不忠实性检测形式化为判别性决策问题,并提供了FINE-CoT(Chain-of-Thought忠实性实例评估),包含由四种代表性LLM在四个领域生成的超过1000条推理轨迹,其中超过300条为不忠实实例,且带有细粒度成因与步骤级证据。我们进一步系统评估了涵盖反事实、基于对数几率和大模型作为裁判三类范式的11种代表性检测方法,获得实证洞察,厘清现有方法的优劣,并揭示在知识密集型领域及更先进模型下检测难度显著上升。据我们所知,FaithCoT-Bench是首个全面的实例级CoT忠实性基准,为未来实现更可解释、更可信的推理研究奠定了坚实基础。
原文摘要 · Abstract (English)
Large language models (LLMs) increasingly rely on Chain-of-Thought (CoT) prompting to improve problem-solving and provide seemingly transparent explanations. However, growing evidence shows that CoT often fail to faithfully represent the underlying reasoning process, raising concerns about their reliability in high-risk applications. Although prior studies have focused on mechanism-level analyses showing that CoTs can be unfaithful, they leave open the practical challenge of deciding whether a specific trajectory is faithful to the internal reasoning of the model. To address this gap, we introduce FaithCoT-Bench, a unified benchmark for instance-level CoT unfaithfulness detection. Our framework establishes a rigorous task formulation that formulates unfaithfulness detection as a discriminative decision problem, and provides FINE-CoT (Faithfulness instance evaluation for Chain-of-Thought), an expert-annotated collection of over 1,000 trajectories generated by four representative LLMs across four domains, including more than 300 unfaithful instances with fine-grained causes and step-level evidence. We further conduct a systematic evaluation of eleven representative detection methods spanning counterfactual, logit-based, and LLM-as-judge paradigms, deriving empirical insights that clarify the strengths and weaknesses of existing approaches and reveal the increased challenges of detection in knowledge-intensive domains and with more advanced models. To the best of our knowledge, FaithCoT-Bench establishes the first comprehensive benchmark for instance-level CoT faithfulness, setting a solid basis for future research toward more interpretable and trustworthy reasoning in LLMs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。