压缩模型的可解释性可通过原模型SAE迁移实现，省去重新训练成本。
On the transferability of Sparse Autoencoders for interpreting compressed models
- 用原模型训练的稀疏自编码器（SAE）可解释压缩后模型。
- 直接剪枝原SAE，性能接近在压缩模型上重训的SAE。
- 适合关注模型压缩与可解释性平衡的研究者。
大型语言模型因规模庞大面临推理效率挑战，压缩方法如剪枝和量化被广泛采用。然而压缩对模型可解释性的影响尚不明确。尽管已有多种解释方法，稀疏自编码器（SAEs）在将模型激活空间分解为特征基方面表现突出。本文研究了原始模型与压缩模型间SAE的差异，发现原模型训练的SAE可有效解释压缩模型，仅略有性能下降；且直接剪枝原SAE，其效果与在压缩模型上重新训练SAE相当。这一发现显著降低了SAE的训练开销。
原文摘要 · Abstract (English)
Modern LLMs face inference efficiency challenges due to their scale. To address this, many compression methods have been proposed, such as pruning and quantization. However, the effect of compression on a model's interpretability remains elusive. While several model interpretation approaches exist, such as circuit discovery, Sparse Autoencoders (SAEs) have proven particularly effective in decomposing a model's activation space into its feature basis. In this work, we explore the differences in SAEs for the original and compressed models. We find that SAEs trained on the original model can interpret the compressed model albeit with slight performance degradation compared to the trained SAE on the compressed model. Furthermore, simply pruning the original SAE itself achieves performance comparable to training a new SAE on the pruned model. This finding enables us to mitigate the extensive training costs of SAEs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。