针对指令模型设计新训练法，让稀疏自编码器更准确还原数据特征。
Breaking the Block: Preserving Data Continuity to Train Superior SAEs for Instruct Models
- 采用序列化训练方式，避免上下文干扰带来的梯度噪声。
- 重建误差降低至0.6468，高质量特征占比达21.1%。
- 适合需要精细控制生成结果的研究者使用。
稀疏自编码器（SAEs）是机制可解释性的核心工具。现有训练方法沿用大语言模型预训练中的块训练范式，导致在指令模型中因注意力泄漏产生破坏性梯度噪声。通过GSNR分析，我们理论揭示该问题，并提出专为指令模型设计的微调对齐序列训练（FAST）方法。FAST使SAE训练与指令模型的数据分布和激活模式一致，显著提升重建保真度与特征可解释性。实验表明，FAST实现更高GSNR，对数尺度均方误差降至0.6468（基线为5.1985），近零Δ损失（-0.51%至0.37%）。在Llama-3.2-3B-it上，生成21.1%高质量特征，远超基线的7.0%与10.2%。进一步发现，通过SAE干预特殊标记激活可提升生成质量，开辟细粒度控制新路径。代码已开源：https://github.com/Geaming2002/FAST。
原文摘要 · Abstract (English)
Sparse Autoencoders (SAEs) are a cornerstone of mechanistic interpretability. Existing training methods inherit the Block Training paradigm from LLM pre-training, which introduces destructive gradient noise in instruct models due to attention leakage from unrelated contexts. Using GSNR analysis, we theoretically characterize this issue and propose Finetuning-aligned Sequential Training (FAST), a sequential training paradigm specifically designed for instruct models. FAST aligns SAE training with the data distribution and activation patterns of instruct models, substantially improving both reconstruction fidelity and feature interpretability. Experimental results show that FAST achieves higher GSNR, a significantly lower log-scaled MSE of 0.6468 compared to the baseline's 5.1985, and a near-zero Delta Loss (-0.51\% to 0.37\%). Moreover, on Llama-3.2-3B-it, FAST produces 21.1\% high-quality features, substantially outperforming baseline methods that achieve 7.0\% and 10.2\%. We further find that intervening on special token activations through SAEs can improve generation quality, revealing new opportunities for fine-grained control. Our codes are available as open source at https://github.com/Geaming2002/FAST.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。