用有限状态机解析大模型的推理路径,实现高效精准控制。
Modeling Hierarchical Thinking in Large Reasoning Models
- 将复杂推理过程建模为六种认知状态间的有限状态转移。
- 通过状态拓扑分析识别有效与失败推理链,提升准确性。
- 无需训练,在推理时用稀疏激活引导,25倍减少干预次数。
大型推理模型(LRM)通过生成长序列思维链(CoT)解决复杂任务,但其推理轨迹的涌现动态尚不清晰,易导致不一致和推理异常。本文提出将LRM的层次化推理动态近似为在六个抽象认知状态间转移的有限状态机(FSM)轨迹,并证明这些状态与转移可被模型隐状态捕捉。该表示可用于解释与优化。例如,通过分析状态转移拓扑,我们识别出推理策略的统计变化,有助于区分有效与失败的推理链。为此,我们提出无需训练的推理时控制方法——Q-值引导引导(Q-Value guided steering),将推理视为规划问题。通过估计状态转移的长期效用,在句边界施加稀疏正交激活引导,使思维链生成对齐最优推理策略。在四个基准测试(AIME25、MATH-500、GSM8k、GPQA Diamond)上,使用三种前沿开源推理模型进行实验,结果表明,该方法显著提升性能且效率极高,通常只需基线方法25倍少的干预次数,说明通过引导高层次认知动态可高效控制推理过程。代码已公开:https://github.com/shahariar-shibli/CoT-FSM。
原文摘要 · Abstract (English)
Large Reasoning Models (LRMs) solve complex tasks by generating long Chain-of-Thought (CoT) sequences; however, the emergent dynamics governing reasoning trajectories are not well understood and can lead to inconsistencies and reasoning pathologies. In this work, we propose to approximate LRM's emerging hierarchical reasoning dynamics as a trajectory within a Finite State Machine (FSM) transitioning among six abstract cognitive states. We demonstrate that these states and transitions can be captured in the latent state of the model. We believe that this representation can have different applications in the interpretability and optimization of LRM models. For example, by analyzing the topology of these transitions, we identify statistical shifts in reasoning strategies that help identify effective reasoning chains from those that fail. To illustrate these potential advantages, we propose Q-Value guided steering, a training-free inference-time control method that treats reasoning as a planning problem. We estimate the long-horizon utility of state transitions and apply sparse, orthogonal activation steering at sentence boundaries to align the CoT generation with optimal reasoning policies. Experiments across four benchmarks (AIME25, MATH-500, GSM8k, and GPQA Diamond) using three state-of-the-art open reasoning models demonstrate that Q-Value steering policy achieves significant performance gains with "surgical" efficiency, often requiring 25 times fewer interventions than greedy and weighted baselines, which suggests that reasoning can be effectively controlled by guiding high-level cognitive dynamics rather than micro-managing token generation. Code is available at: https://github.com/shahariar-shibli/CoT-FSM.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。