让大模型的推理理由与其真实决策保持一致,提升解释可信度。
Aligning What LLMs Do and Say: Towards Self-Consistent Explanations
- 构建大规模基准PSCB,关联模型决策与多种解释及归因向量。
- 发现斯皮尔曼相关性比余弦相似度更能反映解释与决策的一致性。
- 用直接偏好优化改进解释一致性,不降低任务准确率,适合可解释性研究者。
大型语言模型(LLMs)看似为可解释性提供了便捷路径:只需要求它们解释自己的答案。然而,驱动答案的特征往往与其解释中强调的特征不同,导致事后推理可能歪曲模型输出的真实依据。我们通过比较答案与解释的特征重要性分布来量化这一差距。先前分析已揭示此类差异,但受限于归因方法的高计算成本,大规模研究难以开展。为此,我们提出后验自一致性银行(PSCB),一个大规模基准,将模型决策与跨数据集、方法和模型家族的多样化解释及归因向量相链接。利用PSCB,我们发现斯皮尔曼等级相关性比余弦相似度提供更可靠的对齐信号。基于此洞察,我们采用直接偏好优化(DPO)处理基于归因的偏好数据,提升了对齐性且未损害任务准确性;而标准监督微调在同一数据上无法达到类似效果。这些改进在多个领域具有鲁棒性,为实现可扩展且忠实的模型决策与自然语言解释对齐铺平了道路。
原文摘要 · Abstract (English)
Large language models (LLMs) seem to offer an easy path to interpretability: just ask them to explain their answers. Yet the features driving an answer often differ from those emphasized in its explanation, meaning post-hoc rationales can misrepresent what actually shaped the model's output. We quantify this gap by comparing the feature-importance distributions of answers and their explanations. Prior analyses reveal such discrepancies, but large-scale study has been limited by the high computational cost of attribution methods. To address this, we introduce the Post-hoc Self-Consistency Bank (PSCB), a large-scale benchmark linking model decisions with diverse explanations and attribution vectors across datasets, methods, and model families. Using PSCB, we find that Spearman rank correlation provides a more reliable signal of alignment than cosine similarity. Building on this insight, we apply Direct Preference Optimization (DPO) to attribution-based preference data, improving alignment without degrading task accuracy, and show that standard supervised fine-tuning on the same data fails to achieve comparable gains. These improvements generalize robustly across domains, paving the way toward scalable and faithful alignment between LLM decisions and their natural language explanations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。