通过干预隐空间特征，无需提示即可触发大模型推理行为。
Triggering Chain-of-Thought via Latent Feature Interventions in Large Language Models
- 用稀疏自编码器识别与推理相关的少量隐变量特征。
- 仅干预1-2个特征即实现接近提示推理的准确率。
- 适合研究模型内部机制或优化推理效率的学者。
链式思维（CoT）提示能提升大语言模型的推理性能，但其内部触发信号仍不明确。我们利用稀疏自编码器（SAEs）捕捉的稀疏特征，提出一套系统框架，分析并干预模型内部表示，识别出一组与推理行为相关联的少量隐变量特征，并可通过定向干预进行因果验证。在多个模型家族和推理基准上，我们发现仅引导一个或少数几个推理相关隐变量特征，即可显著诱发推理行为，且无需显式CoT提示，准确率可媲美传统CoT。进一步实验表明，这些特征不依赖特定措辞或冗长表达，且在抑制实验中即便在CoT提示下仍会损害性能，证实其在推理中的因果作用。结果表明，CoT提示通过激活特定隐变量特征来触发推理，而对这些特征的定向干预为无需显式提示的高效推理提供了新路径。代码已开源。
原文摘要 · Abstract (English)
Chain-of-Thought (CoT) prompting often improves the reasoning performance of large language models (LLMs), but the internal signal that triggers this behavior remains poorly understood. Leveraging the sparse features captured by Sparse Autoencoders (SAEs), we propose a systematic framework to analyze and intervene on the internal representations of LLMs, identifying a small set of latent features that are linked to reasoning behavior and can be causally tested through targeted intervention. Across multiple model families and reasoning benchmarks, we show that steering one or a small number of reasoning-related latent features can substantially induce reasoning behavior without explicit CoT prompting, achieving accuracy comparable to CoT. We further show that the identified features are not tied to particular wording patterns or verbosity, and confirm their causal role in reasoning through suppression experiments that impair performance even under CoT prompting. These results suggest that CoT prompting activates specific latent features to trigger reasoning, and that targeted intervention on these features offers an alternative pathway to elicit efficient reasoning behavior without explicit CoT prompting. Code is available at https://github.com/Zhenghao-He/LatentCoT.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。