arXiv:2606.03965cs.CLcs.AI2026-06

用智能控制器动态引导大模型推理,省token还可控。

Agentic Chain-of-Thought Steering for Efficient and Controllable LLM Reasoning

论文配图:Agentic Chain-of-Thought Steering for Efficient and Controllable LLM Reasoning
图 1 · 摘自论文原文
  • 引入控制器代理,根据剩余计算预算动态调整推理策略。
  • 在多个基准上实现准确率相近但节省超40%推理令牌。
  • 适合需要高效可控推理的场景,如资源受限部署。

大型语言模型通过扩展思维链提升最终答案准确率,但常消耗过多令牌且缺乏推理时控制能力。现有高效推理方法通过缩短、提前终止或压缩推理路径控制思考长度,但未明确模型如何思考。本文提出智能思维链引导(ACTS),将推理引导建模为马尔可夫决策过程:一个控制器代理在推理过程中动态引导冻结的推理模型。每一步,控制器观察当前推理轨迹与剩余思考预算,生成包含推理策略和引导语句的行动指令,以启动下一步推理。该方法实现预算感知的策略控制,同时保持推理生成连贯性。控制器通过构建的合成引导轨迹初始化,并结合预算条件奖励函数进行强化学习优化。多基准实验表明,ACTS在保持竞争性准确率的同时显著减少令牌消耗,且支持不同推理模型与任务间的可控精度-令牌权衡。代码已开源。

原文摘要 · Abstract (English)

Large language models improve final-answer accuracy through extended chain-of-thought reasoning, but often spend tokens inefficiently and offer little inference-time control. Existing efficient reasoning methods control thinking length by shortening, early-stopping, or compressing traces, leaving how the model thinks implicit. In this paper, we propose Agentic Chain-of-Thought Steering (ACTS), which formulates reasoning steering as a Markov decision process where a controller agent adaptively steers a frozen reasoner during inference. At each step, the controller observes the reasoning trace and remaining thinking budget, then issues a steering action consisting of a reasoning strategy and a steering phrase that initiates the next reasoner step. This enables budget-aware strategy control for efficient reasoning while preserving the reasoner's generation continuity. We initialize the controller agent from our constructed synthetic steering trajectories with multi-budget augmentation, and further optimize it via reinforcement learning with budget-conditioned reward shaping. Experiments across multiple benchmarks show that ACTS achieves competitive accuracy with substantial token savings, and enables controllable accuracy-token trade-offs across different reasoners and tasks. The code is available at https://github.com/Andree-9/ACTS.

推理优化可控生成强化学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。