用稀疏自编码器预测大模型微调后的跨域表现,无需实际训练。
SAE as a Crystal Ball: Interpretable Features Predict Cross-domain Transferability of LLMs without Training
- 通过稀疏自编码器捕捉模型特征变化,计算其与下游任务的相关性。
- 预测准确率相关系数超0.7,能提前评估微调后性能表现。
- 结果可解释,适合优化大模型后训练策略的研究者使用。
近年来,预训练大语言模型在各类任务中取得显著成功。除自监督预训练外,其在下游应用中的效果还高度依赖于后训练过程——即通过特定数据和目标调整模型。然而,这一过程会引入模型偏移,影响其在不同领域的迁移能力,而这种偏移的迁移机制仍不明确。为揭开黑箱,我们提出基于稀疏自编码器(SAE)的迁移能力评分(STS),用于预测后训练阶段的跨域迁移表现。以监督微调为例,STS识别SAE表征中发生偏移的维度,并计算其与下游领域间的相关性,从而在微调前可靠估计迁移能力。多模型、多领域的实验证明,STS与实际性能变化的皮尔逊相关系数均超过0.7。此外,我们初步探索将STS扩展至强化学习场景。我们认为,STS可作为大模型后训练策略的可解释指导工具。代码已开源:https://github.com/PKU-ML/STS。
原文摘要 · Abstract (English)
In recent years, pre-trained large language models have achieved remarkable success across diverse tasks. Besides the pivotal role of self-supervised pre-training, their effectiveness in downstream applications also depends critically on the post-training process, which adapts models to task-specific data and objectives. However, this process inevitably introduces model shifts that can influence performance in different domains, and how such shifts transfer remains poorly understood. To open up the black box, we propose the SAE-based Transferability Score (STS), a new metric that leverages sparse autoencoders (SAEs) to forecast post-training transferability. Taking supervised fine-tuning as an example, STS identifies shifted dimensions in SAE representations and calculates their correlations with downstream domains, enabling reliable estimation of transferability \textit{before} fine-tuning. Extensive experiments across multiple models and domains show that STS accurately predicts the transferability of supervised fine-tuning, achieving Pearson correlation coefficients above 0.7 with actual performance changes. Beyond this, we take an initial step toward extending STS to reinforcement learning. We believe that STS can serve as an {\color{black} interpretable} tool for guiding post-training strategies in LLMs. Code is available at https://github.com/PKU-ML/STS.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。