用激活补丁技术评估大模型解释的可信度，发现对齐训练提升解释质量。
Towards Faithful Natural Language Explanations: A Study Using Activation Patching in Large Language Models
- 通过激活补丁追踪模型内部因果路径，衡量解释与答案的一致性。
- 2B到27B参数模型中，对齐训练后模型解释更可信、更合理。
- 避免分布外样本干扰，适合研究模型可解释性与对齐机制的人参考。
大型语言模型（LLMs）能生成令人信服的自然语言解释（NLEs）来支持其回答，但这些解释的可信度不应轻易相信。现有评估方法通常在解释或特征层面引入扰动，但这些方法既不全面也不符合可信度的定义。我们指出，基于分布外样本的可信度结论存在风险。本文采用因果中介技术——激活补丁，测量解释对支撑答案的因果一致性。提出的新指标Causal Faithfulness以因果归因的一致性作为可信度依据。实验涵盖2B至27B参数的多种模型，发现经过对齐训练的模型生成的解释更具可信度和合理性。相较于现有方法，该指标考虑了模型内部计算过程，避免了分布外问题，提升了评估有效性。代码已开源：https://github.com/wj210/Causal-Faithfulness。
原文摘要 · Abstract (English)
Large Language Models (LLMs) are capable of generating persuasive Natural Language Explanations (NLEs) to justify their answers. However, the faithfulness of these explanations should not be readily trusted at face value. Recent studies have proposed various methods to measure the faithfulness of NLEs, typically by inserting perturbations at the explanation or feature level. We argue that these approaches are neither comprehensive nor correctly designed according to the established definition of faithfulness. Moreover, we highlight the risks of grounding faithfulness findings on out-of-distribution samples. In this work, we leverage a causal mediation technique called activation patching, to measure the faithfulness of an explanation towards supporting the explained answer. Our proposed metric, Causal Faithfulness quantifies the consistency of causal attributions between explanations and the corresponding model outputs as the indicator of faithfulness. We experimented across models varying from 2B to 27B parameters and found that models that underwent alignment tuning tend to produce more faithful and plausible explanations. We find that Causal Faithfulness is a promising improvement over existing faithfulness tests by taking into account the model's internal computations and avoiding out of distribution concerns that could otherwise undermine the validity of faithfulness assessments. We release the code in \url{https://github.com/wj210/Causal-Faithfulness}
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。