用对比归因分析大模型在真实评测中的失败原因
Contrastive Attribution in the Wild: An Interpretability Analysis of LLM Failures on Realistic Benchmarks
- 提出跨层对比归因方法，定位错误输出的输入与内部状态影响
- 在多个数据集和模型规模上验证，发现该方法在部分错误中有效
- 适用于调试大模型真实场景缺陷，尤其适合关注可解释性的研究者
可解释性工具被越来越多用于分析大语言模型（LLM）的失败，但以往研究多集中于短提示或简化场景，对常用基准测试中的表现关注不足。为此，本文研究基于LRP的对比归因作为一种实用工具，在真实场景下分析LLM失败行为。我们将失败分析定义为对比归因，将错误输出词元与正确替代词元之间的逻辑值差异归因于输入词元和模型内部状态，并引入一种高效扩展，支持长上下文输入的跨层归因图构建。利用该框架，我们在多个基准上进行了系统性实证研究，比较不同数据集、模型规模和训练检查点下的归因模式。结果表明，该词元级对比归因在某些失败案例中能提供有效信号，但并非普遍适用，揭示了其在真实场景中分析大模型失败时的潜力与局限。代码已公开：https://aka.ms/Debug-XAI。
原文摘要 · Abstract (English)
Interpretability tools are increasingly used to analyze failures of Large Language Models (LLMs), yet prior work largely focuses on short prompts or toy settings, leaving their behavior on commonly used benchmarks underexplored. To address this gap, we study contrastive, LRP-based attribution as a practical tool for analyzing LLM failures in realistic settings. We formulate failure analysis as \textit{contrastive attribution}, attributing the logit difference between an incorrect output token and a correct alternative to input tokens and internal model states, and introduce an efficient extension that enables construction of cross-layer attribution graphs for long-context inputs. Using this framework, we conduct a systematic empirical study across benchmarks, comparing attribution patterns across datasets, model sizes, and training checkpoints. Our results show that this token-level contrastive attribution can yield informative signals in some failure cases, but is not universally applicable, highlighting both its utility and its limitations for realistic LLM failure analysis. Our code is available at: https://aka.ms/Debug-XAI.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。