SFT看似不动模型结构，实则悄悄改了语义特征分布。
A Mechanistic Investigation of Supervised Fine Tuning
- 用预训练稀疏自编码器解析激活变化，发现表征分化
- 发现任务和层特定的语义特征被系统性调整
- 揭示安全对齐有独特层间更新模式，适合模型可解释性研究
大语言模型在监督微调（SFT）前后隐藏层激活的余弦相似度仍很高，表面看模型表征几何未变。但通过将前后激活投影至基座模型上预训练的稀疏自编码器（SAE）中，发现底层稀疏潜在表示显著偏离。本文提出一种新分析流程，利用预训练SAE作为高分辨率诊断工具，深入探究这种表征分化的机制。分析发现，不同任务与层存在特定的语义特征被系统性修改；此外，还识别出与安全对齐相关联的、具有层级特性的参数更新模式。所有代码、实验脚本与分析文件均已公开于：https://github.com/ruhzi/sae-investigation。
原文摘要 · Abstract (English)
The cosine similarity between a large language model's hidden activations before and after Supervised Fine-Tuning (SFT) remains very high. This, at first glance, suggests that SFT leaves the model's activation geometry largely undisturbed. However, projecting both sets of activations through a Sparse Autoencoder (SAE) pretrained on the base model reveals that the underlying sparse latents diverge significantly. We introduce a novel investigative pipeline which utilizes these pretrained SAEs as a high-resolution diagnostic tool to mechanistically investigate the drivers of this representational divergence. Through our analytical pipeline, we discover task-specific and layer-specific distributions of the precise semantic features that are systematically altered during supervised fine-tuning. We additionally identify a layer-wise update profile specific to safety alignment. All code, experimental scripts, and analysis files associated with this work are publicly available at: https://github.com/ruhzi/sae-investigation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。