通过联合训练提升稀疏自编码器特征的原子性,让每个特征只对应一个语义概念。
MetaSAEs: Joint Training with a Decomposability Penalty Produces More Atomic Sparse Autoencoder Latents
- 设计元自编码器,惩罚主模型解码方向可被简单压缩,迫使特征独立
- 在GPT-2大模型上使特征原子性指标提升7.5%,可解释性评分提高7.6%
- 适合关注模型内部表征可解释性与安全对齐的研究者
稀疏自编码器(SAEs)在对齐检测和模型操控等安全相关任务中日益重要,要求其隐变量尽可能原子化——即每个隐变量应仅表示一个单一、连贯的概念,来自同一潜在表示子空间。现实中,隐变量常混合多个子空间,导致单个特征在语义迥异的上下文中激活,干扰模型计算的理解。本文提出一种联合训练目标,直接惩罚这种子空间混杂:训练一个小型元自编码器,以稀疏方式重构主自编码器的解码列;当主模型的解码方向容易被元字典重建时,便施加惩罚——这表明该方向位于其他主方向张成的子空间中。该机制促使解码方向更相互独立,抵抗稀疏压缩。在GPT-2 Large(第20层)上,所选配置使平均|φ|降低7.5%(相对),自动化可解释性(模糊测试)得分提升7.6%,提供独立验证。重建开销较小。在Gemma 2 9B上的结果具方向性;对于未完全收敛的SAE,相同参数设置获得+8.6% ΔFuzz,表明方法具备向更大模型迁移的潜力。定性分析显示,原属多义性的特征被拆分为语义分明的子特征,各自专注于不同表示子空间。
原文摘要 · Abstract (English)
Sparse autoencoders (SAEs) are increasingly used for safety-relevant applications including alignment detection and model steering. These use cases require SAE latents to be as atomic as possible. Each latent should represent a single coherent concept drawn from a single underlying representational subspace. In practice, SAE latents blend representational subspaces together. A single feature can activate across semantically distinct contexts that share no true common representation, muddying an already complex picture of model computation. We introduce a joint training objective that directly penalizes this subspace blending. A small meta SAE is trained alongside the primary SAE to sparsely reconstruct the primary SAE's decoder columns; the primary SAE is penalized whenever its decoder directions are easy to reconstruct from the meta dictionary. This occurs whenever latent directions lie in a subspace spanned by other primary directions. This creates gradient pressure toward more mutually independent decoder directions that resist sparse meta-compression. On GPT-2 large (layer 20), the selected configuration reduces mean $|φ|$ by 7.5% relative to an identical solo SAE trained on the same data. Automated interpretability (fuzzing) scores improve by 7.6%, providing external validation of the atomicity gain independent of the training and co-occurrence metrics. Reconstruction overhead is modest. Results on Gemma 2 9B are directional. On not-fully-converged SAEs, the same parameterization yields the best results, a $+8.6\%$ $Δ$Fuzz. Though directional, this is an encouraging sign that the method transfers to a larger model. Qualitative analysis confirms that features firing on polysemantic tokens are split into semantically distinct sub-features, each specializing in a distinct representational subspace.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。