arXiv:2608.13337cs.LG2026-08

测量位置影响评估结果,同一潜在表示在不同位置测出的效应差异巨大。

Where You Measure Decides What You Measure: Position Selection in Ablation-Based SAE Evaluation

论文配图:Where You Measure Decides What You Measure: Position Selection in Ablation-Based SAE Evaluation
图 1 · 摘自论文原文
  • 通过统一测量位置消除字典差异带来的偏差
  • 相同潜在表示在不同位置测量时,效果差异可达7.6%以上
  • 适合关注模型可解释性与评估方法严谨性的研究者

稀疏自编码器旨在识别语言模型计算的内容,通常通过关闭潜在变量并观察变化来验证其重要性。但一个潜在变量会在多个词元上激活,而效应需在其中一个词元上测量。常规做法是选择激活最强的位置,这一选择几乎从未被报告,且不由实验者决定,而是由所用词典决定。更换词典后,测量位置随之改变。我们发现这并非细节问题:对比谷歌发布的两个针对同一模型的稀疏自编码器,即使潜在变量解码相似度极高,仍有大量对应项在不同位置进行测量。因此,常规评估中常在不同位置比较不同词典。我们训练六个从同一初始化出发、仅训练选择不同的自编码器,使潜在变量含义一致,结果显示:当所有词典在相同位置测量时,本应归因于词典差异的方差(7.6% 和 11.9%)降至接近零。增加评估数据无法缓解此问题;在十六倍规模的语料库下,词典对测量位置的分歧反而加剧。修正只需一行代码:强制所有评估在同一位置进行。我们提出一种基于擦除的因果指标必须报告测量位置才能跨论文比较,并对五篇已发表论文进行了审计。

原文摘要 · Abstract (English)

Sparse autoencoders are meant to name the things a language model computes, and the usual way to check that a latent matters is to switch it off and see what changes. But a latent fires at many tokens, and the effect has to be measured at one of them. The convention is to measure where the latent fires hardest. That choice is almost never reported, and it is not made by the experimenter: it is made by the dictionary under evaluation. Change the dictionary and the measurement moves to a different token. We show this is not a detail. Take two sparse autoencoders released by Google for the same model and match their latents by decoder similarity: even among the pairs the two dictionaries encode almost identically, they pick different tokens for a large share of them. Two dictionaries compared under the usual protocol are therefore very often compared at different places. To separate the convention from the dictionaries we train six autoencoders from one initialisation, differing only in fitting choices, so that a latent means the same thing in each. Most of the variance such a comparison reads as "these dictionaries disagree about this latent" turns out to be the position instead: it falls from 7.6% and 11.9% of variance to near zero once every dictionary is measured at the same token. More evaluation data does not rescue it. Across a sixteenfold range of corpus sizes the dictionaries agree less about where to measure, not more, so the problem grows with scale. The correction is one line of evaluation code. We give the protocol an ablation-based causal number must report to be comparable across papers, and an audit of five published papers against it. In short: a causal number reported without its position describes the token it was taken at as much as the latent it was taken from.

可解释性模型评估稀疏编码

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。