同一模型不同分析设置下，解释结果差异巨大，无法满足合规要求。
Explanation Multiplicity: Circuit-Level Interpretability Evidence Does Not Survive Defensible Analytic Variation
- 设计7个分析维度的预注册实验，测试解释一致性
- 73.2%的解释声明在不同设置下发生翻转，可信度极低
- 适合关注AI可解释性可靠性与监管合规的研究者
欧盟《人工智能法案》要求高风险系统提供技术文档说明决策机制。机械可解释性是理想证据来源，电路发现是最成熟的工具。我们检验该证据在实际使用场景下的稳定性：两名合格分析师、相同系统、相同工具，仅分析设置不同。预注册了涵盖7个分析维度的交叉网格，每个维度取自已发表实现，通过确定性声明映射将发现的电路转化为结构化附件四声明。在GPT-2 small和间接宾语识别任务上，共15,840项预注册规格中，7,561项生成了声明。其中73.2%（95%置信区间0.725至0.738）的规格对之间声明发生翻转，最常见声明仅覆盖41.1%的空间。所有合规评估机构可能接受的容差下，证据均不满足可提交标准。即使标准化最具影响力的选择——评估指标，翻转率仍达59.4%；若移除电路大小影响并固定其值，翻转率仍为27.1%（95%置信区间0.255至0.286），高于预设阈值。这些电路在结构上几乎不重叠（中位数成对Jaccard重叠仅4%），功能上也无相关性（Cohen's kappa为0.015），说明不稳定性并非同一机制的不同表述。我们提出可提交性标准作为独立协议，并报告该库自身规范任务中一个发现目标根本未执行。本研究仅覆盖一个模型和一个任务，结论是否具普遍性尚待验证。
原文摘要 · Abstract (English)
The EU AI Act requires providers of high-risk systems to file technical documentation describing how the system reaches its decisions. Mechanistic interpretability is the obvious source of such evidence, and circuit discovery is its most developed instrument. We ask whether that evidence survives the condition under which it would be relied upon: two competent analysts, the same system, the same tool, different defensible settings. We pre-registered a crossed grid of seven analytic axes, every level taken from a published implementation, and mapped each discovered circuit through a deterministic claim map to a structured Annex IV statement. Across 15,840 pre-registered specifications on GPT-2 small and the indirect object identification task, of which 7,561 produced a claim, the derived statement flips across 73.2% of specification pairs (95% CI 0.725 to 0.738) and the modal claim commands 41.1% of the space. The evidence fails a filability criterion at every tolerance a conformity assessment body would plausibly accept. Standardising the single most influential choice, the evaluation metric, leaves the flip rate at 59.4%. Removing circuit size from the claim entirely and holding it fixed leaves 27.1% (95% CI 0.255 to 0.286), still above the pre-registered threshold. The circuits underlying these claims are structurally near-disjoint, median pairwise Jaccard overlap 4%, and functionally uncorrelated at Cohen's kappa 0.015, so the instability is not one mechanism described in different words. We give the filability criterion as a standalone protocol, and we report that one of the seven documented discovery objectives does not execute at all on the library's own canonical task. The study covers one model and one task, and whether the conclusion holds at scale is untested.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。