为生成式语言模型设计可解释的归因契约,避免误读自归因陷阱。
The Attribution Contract for Generative Language Models

- 提出归因契约框架,明确每个归因分数对应的解释问题。
- 自归因占比达29%,与输入问题贡献相当,误读是常见谬误。
- 不同生成机制影响归因可靠性,需按场景定制解释方式。
特征归因通过衡量输入各部分对模型输出的解释程度来评估重要性。我们指出,在生成式语言模型中,归因分数并无固定意义:分类器只有一个输出需解释,而生成模型逐词生成输出,每一步生成的词既是输出也是后续输入,导致解释问题变得多重且复杂。为此,我们提出归因契约(Attribution Contract)框架,明确定义归因分数所回答的具体问题,包括被解释的模型得分、固定内容、目标输出、生成过程及可归因特征范围。这些选择在实践中至关重要。例如,在局部下一词契约下,模型自身生成的词获得29%的归因质量,与输入问题的25%相近,若将其误作提示级解释,则陷入‘自归因谬误’。生成过程本身也影响归因效果:在混合专家模型中,集成梯度的完整性不可靠,因沿归因路径重算专家路由会产生显著残差;而固定决策的掩码扩散模型,残差随步数增加而减小。结果表明,为生成式语言模型设计和评估归因方法时,必须明确定义归因契约,因为其含义与评价完全依赖于解释设定。
原文摘要 · Abstract (English)
Feature attribution scores each part of an input by how much it explains a model's output. We argue that in generative language models these scores carry no fixed meaning. A classifier has a single output to explain, but a generative model produces its output token by token, and each generated token is both an output and an input, so explaining the output becomes several distinct questions. We support this claim by proposing the Attribution Contract, a framework that names the question a set of attribution scores answers. A contract specifies the model score being explained, what is held fixed, the target output, the generation process, and which features can receive attribution, choices that matter in practice. For example, eligible features decide what a score explains: under a local next-token contract, the model's own generated tokens receive $29\%$ of the attribution mass, comparable to the $25\%$ on the input question, and reading this as a prompt-level explanation is a mistake we name the self-attribution fallacy. The generative process also shapes feature attribution: on a mixture-of-experts model, Integrated Gradients completeness is unreliable, since recomputing expert routing along the attribution path leaves a large residual that holding the routing fixed substantially lowers, while a masked-diffusion model, which fixes its choices at generation, has a residual that instead decreases with more steps. These results show that Attribution Contracts should be carefully specified when proposing and evaluating attribution methods for generative language models, since the meaning and evaluation of an attribution depend on its explanatory setting.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。