用稀疏自编码器揭示大模型推理时的内部思维机制
I Have Covered All the Bases Here: Interpreting Reasoning Features in Large Language Models via Sparse Autoencoders
- 通过稀疏自编码器提取模型激活中的可解释特征
- 发现不确定、探索与反思类特征在推理中显著活跃
- 适合对模型可解释性与推理机制感兴趣的 researchers
近期的大语言模型（如 DeepSeek-R1）在生成过程中结合深度思考与复杂推理，达到顶尖性能，但其内部推理机制仍不明确。我们观察到这些推理模型持续使用与人类推理过程相关的词汇，推测其对应模型内部特定的推理时刻。为此，我们采用稀疏自编码器（SAEs）对神经网络激活进行稀疏分解，以提取人类可理解的特征。我们提出 ReasonScore，一种自动识别推理阶段活跃 SAE 特征的指标。通过人工与自动分析，发现这些特征与不确定性、探索性思维和反思行为的激活模式一致。控制实验表明，增强这些特征可使模型在推理密集型基准上性能提升 2.2%，推理路径长度增加 20.5%。模型对比分析显示，这些特征仅存在于具备推理能力的模型中。本工作为理解大型语言模型的机械推理机制迈出第一步。代码开源：https://github.com/AIRI-Institute/SAE-Reasoning
原文摘要 · Abstract (English)
Recent LLMs like DeepSeek-R1 have demonstrated state-of-the-art performance by integrating deep thinking and complex reasoning during generation. However, the internal mechanisms behind these reasoning processes remain unexplored. We observe reasoning LLMs consistently use vocabulary associated with human reasoning processes. We hypothesize these words correspond to specific reasoning moments within the models' internal mechanisms. To test this hypothesis, we employ Sparse Autoencoders (SAEs), a technique for sparse decomposition of neural network activations into human-interpretable features. We introduce ReasonScore, an automatic metric to identify active SAE features during these reasoning moments. We perform manual and automatic interpretation of the features detected by our metric, and find those with activation patterns matching uncertainty, exploratory thinking, and reflection. Through steering experiments, we demonstrate that amplifying these features increases performance on reasoning-intensive benchmarks (+2.2%) while producing longer reasoning traces (+20.5%). Using the model diffing technique, we provide evidence that these features are present only in models with reasoning capabilities. Our work provides the first step towards a mechanistic understanding of reasoning in LLMs. Code available at https://github.com/AIRI-Institute/SAE-Reasoning
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。