提出无数据特征对齐方法，让不同层的神经网络特征可比较。
Mechanistic Permutability: Match Features Across Layers
- 通过最小化折叠参数的均方误差实现跨层特征匹配。
- 在Gemma 2上验证了特征在多层间持续存在并能近似隐藏状态。
- 无需训练数据，适用于机制可解释性研究中的特征追踪。
理解深度神经网络中特征在各层间的演变是机制可解释性领域的基础挑战，尤其受多义性和特征叠加的影响。尽管稀疏自编码器（SAEs）可用于提取单层的可解释特征，但跨层对齐仍是一个开放问题。本文提出SAE Match，一种全新的、无需数据的跨层特征对齐方法。该方法通过最小化SAEs折叠参数之间的均方误差实现特征匹配，并将激活阈值融入编码器与解码器权重，以应对特征尺度差异。在Gemma 2语言模型上的大量实验表明，该方法能有效捕捉特征演化过程，显著提升跨层特征匹配质量。我们还发现特征可在多个层间持续存在，且该方法可近似跨层隐藏状态。本工作推进了对神经网络特征动态的理解，为机制可解释性研究提供了新工具。
原文摘要 · Abstract (English)
Understanding how features evolve across layers in deep neural networks is a fundamental challenge in mechanistic interpretability, particularly due to polysemanticity and feature superposition. While Sparse Autoencoders (SAEs) have been used to extract interpretable features from individual layers, aligning these features across layers has remained an open problem. In this paper, we introduce SAE Match, a novel, data-free method for aligning SAE features across different layers of a neural network. Our approach involves matching features by minimizing the mean squared error between the folded parameters of SAEs, a technique that incorporates activation thresholds into the encoder and decoder weights to account for differences in feature scales. Through extensive experiments on the Gemma 2 language model, we demonstrate that our method effectively captures feature evolution across layers, improving feature matching quality. We also show that features persist over several layers and that our approach can approximate hidden states across layers. Our work advances the understanding of feature dynamics in neural networks and provides a new tool for mechanistic interpretability studies.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。