同一数据训练的稀疏自编码器，因随机种子不同而发现不同特征。
Sparse Autoencoders Trained on the Same Data Learn Different Features
- 相同数据与模型下，仅改变权重初始化种子，特征集差异显著。
- 131K隐变量的SAE中，仅30%特征在不同种子间共享。
- 基于TopK的SAE更依赖随机种子，需谨慎解读其发现的特征。
稀疏自编码器（SAEs）是揭示大语言模型（LLMs）激活中人类可理解特征的有效工具。尽管人们预期SAE能发现模型真正使用的底层特征，但我们的研究显示，即使在相同模型和数据上训练，仅因权重初始化的随机种子不同，SAE也会识别出不同的特征集合。例如，在对Llama 3 8B的前馈网络训练的131K隐变量SAE中，不同种子间仅有30%的特征重合。这一现象在三个不同LLM的多个层级、两个数据集及多种SAE架构中均被观察到。虽然使用L1正则化的ReLU SAE在种子间表现出更高稳定性，但采用当前主流TopK激活函数的SAE则更具种子依赖性，即使控制了稀疏度水平也是如此。结果表明，SAE揭示的特征应被视为对激活空间的一种实用分解，而非模型真正使用特征的完整且普适列表。
原文摘要 · Abstract (English)
Sparse autoencoders (SAEs) are a useful tool for uncovering human-interpretable features in the activations of large language models (LLMs). While some expect SAEs to find the true underlying features used by a model, our research shows that SAEs trained on the same model and data, differing only in the random seed used to initialize their weights, identify different sets of features. For example, in an SAE with 131K latents trained on a feedforward network in Llama 3 8B, only 30% of the features were shared across different seeds. We observed this phenomenon across multiple layers of three different LLMs, two datasets, and several SAE architectures. While ReLU SAEs trained with the L1 sparsity loss showed greater stability across seeds, SAEs using the state-of-the-art TopK activation function were more seed-dependent, even when controlling for the level of sparsity. Our results suggest that the set of features uncovered by an SAE should be viewed as a pragmatically useful decomposition of activation space, rather than an exhaustive and universal list of features "truly used" by the model.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。