arXiv:2604.11061cs.LGcs.AI2026-04被引 1

测试解释方法在模型不靠谱时是否有效,发现梯度法更可靠。

Pando: Do Interpretability Methods Work When Models Won't Explain Themselves?

  • 构建新基准Pando,让模型选择真实、虚假或无解释。
  • 当解释不可靠时,梯度法比其他方法提升3-5个百分点。
  • 适合研究模型可解释性与对齐评估的学者参考。

机制可解释性常用于对齐审计,但许多评估未控制仅用黑盒提示能否复现目标行为,导致白盒工具的增益可能来自诱导而非内部信号,此为诱因混淆。我们提出Pando,一种模型生物基准,通过解释轴打破该混淆:模型被训练为对真规则产生忠实解释、不解释或对无关干扰规则产生自信但不忠实的解释。在720个微调模型(实现隐藏决策树规则)上,代理从10个标注的查询-响应对预测模型输出,可选地结合一种可解释性工具输出。当解释忠实,黑盒诱导表现匹配或优于所有白盒方法;当解释缺失或误导时,基于梯度的归因提升准确率3-5个百分点,相关性修补(RelP)带来最大增益,而逻辑透镜、稀疏自编码器和电路追踪均无稳定收益。方差分解表明梯度追踪决策计算,因果驱动输出,而其他读出受任务表征、字段身份和取值偏见主导。所有模型、代码与评估基础设施均已发布。

原文摘要 · Abstract (English)

Mechanistic interpretability is often motivated for alignment auditing, where a model's verbal explanations can be absent, incomplete, or misleading. Yet many evaluations do not control whether black-box prompting alone can recover the target behavior, so apparent gains from white-box tools may reflect elicitation rather than internal signal; we call this the elicitation confounder. We introduce Pando, a model-organism benchmark that breaks this confound via an explanation axis: models are trained to produce either faithful explanations of the true rule, no explanation, or confident but unfaithful explanations of a disjoint distractor rule. Across 720 finetuned models implementing hidden decision-tree rules, agents predict held-out model decisions from $10$ labeled query-response pairs, optionally augmented with one interpretability tool output. When explanations are faithful, black-box elicitation matches or exceeds all white-box methods; when explanations are absent or misleading, gradient-based attribution improves accuracy by 3-5 percentage points, and relevance patching, RelP, gives the largest gains, while logit lens, sparse autoencoders, and circuit tracing provide no reliable benefit. Variance decomposition suggests gradients track decision computation, which fields causally drive the output, whereas other readouts are dominated by task representation, biases toward field identity and value. We release all models, code, and evaluation infrastructure.

可解释性模型对齐基准测试梯度分析

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。