稀疏自编码器在特征相关时会合并相似特征，导致语义不清晰。
Feature Hedging: Correlated Features Break Narrow Sparse Autoencoders
- 通过理论与实验发现特征相关时稀疏编码器会合并特征
- 越窄的编码器越严重，重建损失是主因
- 提出改进型马特约什卡结构，适合大模型可解释性研究
稀疏自编码器（SAEs）通常假设多义激活可分解为可解释的线性方向，前提是激活由底层特征的稀疏线性组合构成。然而我们发现，若SAE宽度小于训练数据中真实特征数量，且特征间存在相关性，则SAE会将相关特征的成分合并，破坏单义性。在大语言模型（LLM）的SAEs中，这两个条件几乎必然成立。这种现象称为特征对冲（feature hedging），由SAE重建损失引发，且越窄的SAE问题越严重。本文系统研究了该现象，涵盖玩具模型的理论分析和基于LLM SAE的实证验证。我们推测特征对冲可能是SAE性能持续低于监督基线的核心原因。最后，基于对对冲机制的理解，提出一种改进的马特约什卡式SAE。重要的是，本工作表明：SAE宽度并非中性超参数，更窄的SAE受对冲影响更大。
原文摘要 · Abstract (English)
It is assumed that sparse autoencoders (SAEs) decompose polysemantic activations into interpretable linear directions, as long as the activations are composed of sparse linear combinations of underlying features. However, we find that if an SAE is more narrow than the number of underlying "true features" on which it is trained, and there is correlation between features, the SAE will merge components of correlated features together, thus destroying monosemanticity. In LLM SAEs, these two conditions are almost certainly true. This phenomenon, which we call feature hedging, is caused by SAE reconstruction loss, and is more severe the narrower the SAE. In this work, we introduce the problem of feature hedging and study it both theoretically in toy models and empirically in SAEs trained on LLMs. We suspect that feature hedging may be one of the core reasons that SAEs consistently underperform supervised baselines. Finally, we use our understanding of feature hedging to propose an improved variant of matryoshka SAEs. Importantly, our work shows that SAE width is not a neutral hyperparameter: narrower SAEs suffer more from hedging than wider SAEs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。