通过激活干预,精准阻止大模型推理中的死循环。
Can We Break LLMs Out of Self-Loops? Fine-Grained Reasoning Control with Activation Steering

- 将推理过程视为隐状态序列,用状态转移构建干预向量库。
- 在推理中实时检测并打破自循环,提升任务准确率与效率。
- 适合需要高可靠推理的场景,如数学证明、复杂决策。
大语言模型的长程推理已成为前沿应用标准,但其推理轨迹难以控制。现有方法依赖提示词,在输入层操作,无法精细调控推理过程本身。已有研究分析了模型推理轨迹中的潜在状态转移动态。本文统计刻画这些状态,发现失败轨迹会陷入自循环,耗尽词元预算却无进展。为此,提出SOPHIA:基于隐藏状态干预与激活的推理过程控制方法。将每个推理前缀分类为隐状态,记录逐步转移,构建按状态对索引的干预向量库。推理时,控制器判断当前状态,根据目标状态检索对应向量,并可在线检测自循环,防止模型陷入推理黑洞。大量实验表明,该方法能有效干预自循环失败,干预向量具备跨状态对泛化能力。最终任务准确率与词元效率均提升,证明精细控制带来更优推理质量。
原文摘要 · Abstract (English)
Extended reasoning has become standard for frontier Large Language Models (LLMs), yet the trajectories these models produce remain largely uncontrollable. Existing methods for shaping how a model reasons are prompt based approaches and operate at the input level, offering no fine-grained control over the reasoning process itself. Related work analyzes and discovers latent transition dynamics in the reasoning traces from Large Language Models. Building on this, we statistically characterize these states, and show that failure trajectories get stuck in self-loops, exhausting the token budget without progress toward the final answer. To intervene on these failures, We propose SOPHIA: Steering Of reasoning Processes via Hidden-state Intervention and Activations. We treat each reasoning trace as a sequence of latent states rather than an unstructured texts, and investigate whether inference time interventions can provide fine-grained control over the self-looping reasoning process. We classify every prefix to a latent state, record step level transitions, and use them to construct a bank of steering vectors indexed by state pairs. At inference time, a controller infers the current state and, given a target state, retrieves the corresponding vector and can also detect self-loops online from the transition structure to prevent the model from sinking into a reasoning black hole. Through extensive experiments, our method reliably intervenes on self-loop failures, with steering vectors that generalize to different state pairs. End task accuracy and token efficiency indicate that fine-grained controllability results in better reasoning quality.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。