通过操控模型隐空间方向，高效控制大模型反思频率。
ReflCtrl: Controlling LLM Reflection Efficiently via Representation Engineering
- 在隐空间找到区分反思与非反思的特定方向。
- 可减少43.2%推理令牌数，且保持准确率不变。
- 适合追求高效推理的开发者与部署者使用。
大型推理模型通过生成长链思维在多种任务上表现优异。自我反思——回顾并修正先前推理步骤的能力——被认为是性能提升的关键因素。然而，自我反思也带来显著的推理开销，其内在机制仍不清晰。本文从表示工程视角研究自我反思：首先，我们识别出模型隐空间中分离反思与非反思步骤的反射方向，并发现沿该方向的激活值能强预测答案正确性，表明自我反思受模型内部不确定性调控。基于此，我们提出ReflCtrl框架，通过分步引导方法控制反思行为：仅在每个新推理步骤开始时施加干预，实现对反思频率的精细控制，且不损害生成质量。在数学与通用推理基准上的实验表明，反思常为冗余，尤其在更强模型中：ReflCtrl在保持准确率的同时，将总推理令牌数减少高达43.2%，且在相同令牌预算下显著优于传统逐标记引导方法。
原文摘要 · Abstract (English)
Large reasoning models achieve strong performance on diverse tasks by producing extended chains of thought. Self-reflection, the ability to review and revise prior reasoning steps, is widely regarded as a key contributor to this performance. However, self-reflection also incurs substantial inference cost, and its governing mechanism remains underexplored. In this work, we study self-reflection through the lens of representation engineering. First, we identify a reflection direction in the model's latent space that separates reflection steps from non-reflection steps, and show that activation along this direction is strongly predictive of answer correctness, suggesting that self-reflection is regulated by the model's internal uncertainty. Next, building on this insight, we propose ReflCtrl, a framework that controls self-reflection via a stepwise steering method: interventions are applied only at the start of each new reasoning step, enabling fine-grained control over reflection frequency without degrading generation quality. Experiments across math and general reasoning benchmarks show that reflection is often redundant, especially in stronger models: ReflCtrl reduces total reasoning tokens by up to 43.2% while preserving accuracy, and substantially outperforms the conventional approach that steers at every token, at matched token budgets.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。