让大模型动态调整内部推理路径,提升准确率并减少计算量。
Internalizing LLM Reasoning via Discovery and Replay of Latent Actions
- 通过发现和重放隐变量中的推理动作,动态控制模型思考过程。
- 在6个基准上平均准确率提升1.9%~7.5%,生成token减少最多35%。
- 适合关注推理效率与模型内部机制的研究者或应用开发者。
将思维链过程内化到隐藏状态已成为高效扩展测试时计算的范式。然而,现有激活调控方法依赖静态控制向量,难以适应复杂推理任务的非平稳演化。为此,我们提出STIR(自我蒸馏的内化推理工具),将推理增强重构为动态隐变量轨迹控制问题。STIR采用三阶段协同流程:(1) 差分内在动作诱导从隐状态中挖掘推理成功经验,提炼调控基元;(2) 稀疏控制基底构建生成紧凑且几何多样化的工具库;(3) 价值调制轨迹干预通过基于锚点的门控机制,动态注入上下文相关的刺激信号。在四个代表性模型上的六个算术与逻辑基准测试中,相比原生解码,STIR使平均准确率提升1.9%至7.5%,平均词元消耗降低最多35%。结果表明,通过动态隐变量轨迹控制,可实现显式思维链的优势,将推理过程内化以规避显式生成,同时保持更高保真度。代码已公开于 https://github.com/sznnzs/LLM-Latent-Action。
原文摘要 · Abstract (English)
The internalization of chain-of-thought processes into hidden states has emerged as a highly efficient paradigm for scaling test-time compute. However, existing activation steering methods rely on static control vectors that fail to adapt to the non-stationary evolution of complex reasoning tasks. To address this limitation, we propose STIR (Self-Distilled Tools for Internal Reasoning), a framework that reformulates reasoning enhancement as a dynamic latent trajectory control problem. STIR introduces a synergistic three-stage pipeline: (1) differential intrinsic action induction harvests latent reasoning successes to crystallize steering primitives; (2) sparse control basis construction curates a compact, geometrically diverse tool library; and (3) value-modulated trajectory intervention dynamically injects context-specific impulses via anchor-based gating. Extensive experiments on six arithmetic and logical benchmarks across four representative models demonstrate that STIR improves average accuracy by 1.9% to 7.5% while reducing average token consumption by up to 35% compared to vanilla decoding. These findings demonstrate that the benefits of explicit chain-of-thought can be realized through dynamic latent trajectory control, internalizing the reasoning process to bypass the explicit generation while achieving superior fidelity. Our code is available at https://github.com/sznnzs/LLM-Latent-Action.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。