arXiv:2411.02664cs.LGcs.AI2024-11NeurIPS被引 5

提出编码解释的定义,揭示其隐藏信息并改进评估方法。

Explanations that reveal all through the definition of encoding

  • 定义编码通过条件依赖识别隐藏信息
  • 证明现有评分无法正确排序非编码解释
  • 提出STRIPE-X新评估方法,适合可解释性研究

特征归因旨在突出驱动预测的关键输入。高质量的解释应保留原始预测能力,因此评估通常基于解释的预测性能。然而,对于一类称为编码解释的解释,评估分数高于其显式内容所暗示的水平。由于缺乏对编码中额外预测力来源的普遍描述,探测编码仍具挑战。本文提出一个基于条件依赖的编码定义,该定义与已有编码实例一致。此定义表明,非编码解释包含所有用于生成解释的有用输入,具备“所见即所得”的透明性,便于使用。进一步证明现有评分(ROAR、FRESH、EVAL-X)无法将非编码解释排在编码解释之上,并提出新的评估方法STRIPE-X以纠正这一问题。通过实证验证理论发现,使用STRIPE-X发现:尽管提示大模型生成非编码解释进行情感分析,其输出仍存在编码现象。

原文摘要 · Abstract (English)

Feature attributions attempt to highlight what inputs drive predictive power. Good attributions or explanations are thus those that produce inputs that retain this predictive power; accordingly, evaluations of explanations score their quality of prediction. However, evaluations produce scores better than what appears possible from the values in the explanation for a class of explanations, called encoding explanations. Probing for encoding remains a challenge because there is no general characterization of what gives the extra predictive power. We develop a definition of encoding that identifies this extra predictive power via conditional dependence and show that the definition fits existing examples of encoding. This definition implies, in contrast to encoding explanations, that non-encoding explanations contain all the informative inputs used to produce the explanation, giving them a "what you see is what you get" property, which makes them transparent and simple to use. Next, we prove that existing scores (ROAR, FRESH, EVAL-X) do not rank non-encoding explanations above encoding ones, and develop STRIPE-X which ranks them correctly. After empirically demonstrating the theoretical insights, we use STRIPE-X to show that despite prompting an LLM to produce non-encoding explanations for a sentiment analysis task, the LLM-generated explanations encode.

可解释性特征归因大模型评估方法

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。