提出二值自编码器，让大模型特征更稀疏可解释
Binary Autoencoder for Mechanistic Interpretability of Large Language Models
- 用批量最小熵约束激活值，强制特征稀疏独立
- 二值化激活后计算熵，揭示模型推理动态
- 相比基线方法，减少冗余特征，提升可解释特征数量
现有工作致力于从大语言模型的隐藏状态中解耦出离散的数值组件（特征）。然而，它们通常依赖于训练时对单个样本施加正则化约束的自编码器，缺乏跨实例的全局稀疏性保证，导致大量密集（同时不活跃）特征存在，损害了特征稀疏性和原子化。本文提出一种新型自编码器变体，在小批量隐藏激活上施加最小熵约束，从而促进特征在实例间保持独立与稀疏。为高效计算熵，我们通过阶跃函数将隐藏激活离散化为1比特，并采用梯度估计实现反向传播，因此称之为二值自编码器（BAE）。我们实证展示了两个主要应用：(1) 特征集熵计算。可在二值隐藏激活上可靠估计熵，用于刻画大模型的推理动态；(2) 特征解耦。相比典型方法，由于改进的训练策略，BAE避免了密集特征，同时在基线中产生最多的可解释特征。
原文摘要 · Abstract (English)
Existing works are dedicated to untangling atomized numerical components (features) from the hidden states of Large Language Models (LLMs). However, they typically rely on autoencoders constrained by some training-time regularization on single training instances, without an explicit guarantee of global sparsity among instances, causing a large amount of dense (simultaneously inactive) features, harming the feature sparsity and atomization. In this paper, we propose a novel autoencoder variant that enforces minimal entropy on minibatches of hidden activations, thereby promoting feature independence and sparsity across instances. For efficient entropy calculation, we discretize the hidden activations to 1-bit via a step function and apply gradient estimation to enable backpropagation, so that we term it as Binary Autoencoder (BAE) and empirically demonstrate two major applications: (1) Feature set entropy calculation. Entropy can be reliably estimated on binary hidden activations, which can be leveraged to characterize the inference dynamics of LLMs. (2) Feature untangling. Compared to typical methods, due to improved training strategy, BAE avoids dense features while producing the largest number of interpretable ones among baselines.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。