arXiv:2608.25542cs.LGcs.CL2026-08

通过解耦反射与推理激活,减少大模型推理冗余,提升效率。

Reflection Steering: Disentangling Reflection from Reasoning in Activation Space for Token-Efficient Inference

  • 分离反射与一般推理的隐藏状态,用PCA去噪并正交化方向
  • 在六组对比实验中平均减少16.9%的推理令牌消耗
  • 支持运行时调节强度参数,平衡效率、准确率与生成稳定性

大型推理模型常产生包含验证、修正和回溯的推理轨迹。当反射仅重复检查已确立结果时,会浪费推理令牌并增加延迟。现有反射控制方法多在预设层添加基于标签的均值差方向,但其与推理及长度信号纠缠,导致精度-效率权衡不稳定。本文提出Reflection Steering,一种无需训练的框架,通过在激活空间中解耦反射相关计算来控制大语言模型中的反射行为。具体地,在每层对比有无反射的隐藏状态,利用PCA去噪并正交化反射方向,同时抑制早期干预带来的下游放大效应。通过小样本校准多干预强度,保留稳定层,并对残差流激活应用有界投影移除。在两个公开基准和三个开源权重大模型上进行大量实验,结果表明,Reflection Steering在六组匹配设置中平均减少16.9%的推理令牌。此外,该方法引入有界反射干预强度参数α,可在部署时动态调节以平衡令牌节省、准确率与生成稳定性。

原文摘要 · Abstract (English)

Large reasoning models often produce reasoning traces with verification, revision, and backtracking. When reflection merely re-checks established results, it wastes reasoning tokens and increases latency. Most existing reflection steering methods add a label-derived mean-difference direction across preset layers, but its entanglement with reasoning and length signals destabilizes the accuracy-efficiency trade-off. In this paper, we propose Reflection Steering, a training-free framework for controlling reflection-associated computation within LLMs by disentangling reflection-related activations from general reasoning. Specifically, we contrast reflective and non-reflective hidden states at each LLM layer, denoise the resulting reflection directions with PCA, and orthogonalize them against general-reasoning directions. To limit downstream amplification from early-layer interventions, we calibrate each layer across multiple intervention strengths on a small set, retain only stable layers, and apply bounded projection removal to their residual-stream activations. We conduct extensive experiments across two public benchmarks and three open-weight LLMs against state-of-the-art activation-steering baselines. Results show that Reflection Steering reduces reasoning tokens by 16.9% on average across six matched settings. Besides, our method further introduces a bounded reflection intervention-strength parameter $α$, enabling deployment-time adjustment to balance token savings, accuracy, and generation stability.

大模型优化推理效率激活控制

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。