L0设置不当会导致稀疏自编码器提取错误特征，影响可解释性。
Sparse but Wrong: Incorrect L0 Leads to Incorrect Features in Sparse Autoencoders
- 通过调节L0控制每令牌激活特征数，影响特征解耦效果。
- 过低或过高L0都会导致特征混合，无法分离出单一语义特征。
- 提出代理指标辅助选择合适L0，适用于模型可解释性研究者。
稀疏自编码器（SAE）从大语言模型（LLM）内部激活中提取特征，旨在对应可解释概念。核心训练超参数为L0：每个令牌平均激活的SAE特征数量。现有工作通过稀疏性-重建权衡图比较SAE算法，暗示L0是无正确值的自由参数。本文研究L0对SAE的影响，发现若未正确设置，SAE将无法解耦LLM的底层特征：当L0过低时，模型混合相关特征以提升重建；当L0过高时，出现退化解仍混合特征。我们提出一个代理指标，可指导在特定训练分布下寻找正确L0。该方法在玩具模型中有效，并与LLM SAE的峰值稀疏探测性能一致。多数常用SAE的L0设置过低。结果表明，训练具有单义特征的SAE需正确设定L0。
原文摘要 · Abstract (English)
Sparse Autoencoders (SAEs) extract features from LLM internal activations, meant to correspond to interpretable concepts. A core SAE training hyperparameter is L0: how many SAE features should fire per token on average. Existing work compares SAE algorithms using sparsity-reconstruction tradeoff plots, implying L0 is a free parameter with no inherently correct value aside from its effect on reconstruction. In this work we study the effect of L0 on SAEs, and show that if L0 is not set correctly, the SAE fails to disentangle the underlying features of the LLM. If L0 is too low, the SAE will mix correlated features to improve reconstruction. If L0 is too high, the SAE finds degenerate solutions that also mix features. Further, we present a proxy metric that can help guide the search for the correct L0 for an SAE on a given training distribution. We show that our method finds the correct L0 in toy models and coincides with peak sparse probing performance in LLM SAEs. We find that most commonly used SAEs have an L0 that is too low. Our work shows that practitioners must set L0 correctly to train SAEs with monosemantic features.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。