用特征代替神经元，让大模型知识更清晰可解释。
The Knowledge Microscope: Features as Better Analytical Lenses than Neurons
- 用稀疏自编码器将神经元分解为更精细的特征
- 特征能更好表达知识，且对隐私信息删除更有效
- 适合关注模型可解释性与数据隐私的研究者
以往研究多以MLP神经元为分析单元理解语言模型中的事实知识，但神经元存在多义性问题，导致知识表达有限且可解释性差。本文通过初步实验验证，稀疏自编码器（SAE）能有效将神经元分解为特征，作为替代分析单元。核心发现显示：(1) 特征对知识表达影响更强，可解释性更优；(2) 特征具备更强的单义性，相关与无关事实间激活模式差异显著；(3) 特征在隐私保护上优于神经元，通过提出的FeatureEdit方法，能更有效擦除语言模型中的敏感信息，显著超越现有基于神经元的方法。代码与数据集将公开。
原文摘要 · Abstract (English)
Previous studies primarily utilize MLP neurons as units of analysis for understanding the mechanisms of factual knowledge in Language Models (LMs); however, neurons suffer from polysemanticity, leading to limited knowledge expression and poor interpretability. In this paper, we first conduct preliminary experiments to validate that Sparse Autoencoders (SAE) can effectively decompose neurons into features, which serve as alternative analytical units. With this established, our core findings reveal three key advantages of features over neurons: (1) Features exhibit stronger influence on knowledge expression and superior interpretability. (2) Features demonstrate enhanced monosemanticity, showing distinct activation patterns between related and unrelated facts. (3) Features achieve better privacy protection than neurons, demonstrated through our proposed FeatureEdit method, which significantly outperforms existing neuron-based approaches in erasing privacy-sensitive information from LMs.Code and dataset will be available.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。