arXiv:2604.02113cs.CL2026-04被引 1

提出稳定筛选方法,提升大模型推理控制的可靠性。

Reliable Control-Point Selection for Steering Reasoning in Large Language Models

  • 基于概率模型识别推理行为的稳定性,过滤无效控制点。
  • 在MATH-500上达0.784准确率,优于最强基线5.0个百分点。
  • 适合需要可靠推理控制的模型微调与跨模型迁移场景。

控制向量为大模型推理行为提供无训练调控机制,但有效构建依赖于从隐藏状态中识别真实行为信号。对于可通过提示切换的行为,检测较简单;但自省等自发性推理行为难以通过提示控制。现有方法依赖思维链中的关键词匹配来检测行为边界,隐含假设所有检测边界均代表真实信号。我们发现该假设严重错误:在541个关键词检测边界中,93.3%行为不稳定,相同前缀重生成时无法重现目标行为。为此,我们建立概率模型,将内在推理行为视为上下文依赖的随机事件,指出不稳定的边界会稀释控制信号。基于此,提出稳定性过滤,仅保留行为一致的边界。结合内容子空间投影消除问题相关噪声,新方法在MATH-500上达到0.784准确率(比最强基线提升5.0)。所得控制向量可跨同架构模型迁移,提升Nemotron-Research-Reasoning-1.5B(+5.0)和DeepScaleR-1.5B-Preview(+6.0)。代码已开源。

原文摘要 · Abstract (English)

Steering vectors offer a training-free mechanism for controlling reasoning behaviors in large language models, but constructing effective vectors requires identifying genuine behavioral signals in the model's hidden states. For behaviors that can be toggled via prompts, this is straightforward. However, many reasoning behaviors -- such as self-reflection -- emerge spontaneously and resist prompt-level control. Current methods detect these behaviors through keyword matching in chain-of-thought traces, implicitly assuming that every detected boundary encodes a genuine behavioral signal. We show that this assumption is overwhelmingly wrong: across 541 keyword-detected boundaries, 93.3\% are behaviorally unstable, failing to reproduce the detected behavior under re-generation from the same prefix. We develop a probabilistic model that formalizes intrinsic reasoning behaviors as stochastic events with context-dependent trigger probabilities, and show that unstable boundaries dilute the steering signal. Guided by this analysis, we propose stability filtering, which retains only boundaries where the model consistently reproduces the target behavior. Combined with a content-subspace projection that removes residual question-specific noise, our method achieves 0.784 accuracy on MATH-500 (+5.0 over the strongest baseline). The resulting steering vectors transfer across models in the same architecture family without re-extraction, improving Nemotron-Research-Reasoning-1.5B (+5.0) and DeepScaleR-1.5B-Preview (+6.0). Code is available at https://github.com/zhmzm/stability-steering.

推理控制稳定性筛选大模型控制向量

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。