arXiv:2603.15051cs.CLcs.AI2026-03中稿 · ICLR被引 3

让大模型在隐空间中自适应推理,更省力更准确。

Thinking in Latents: Adaptive Anchor Refinement for Implicit Reasoning in LLMs

  • 用可迭代优化的隐向量锚点替代显式思维链
  • 自适应终止机制使难题多算、易题少算,平均减少50%计算步数
  • 相比传统方法减少92%输出 token,适合高效推理场景

基于词元级思维链(CoT)提示已成为激发大语言模型多步推理的标准方法,尤其适用于数学应用题。然而,生成长中间推理路径会增加输出长度与推理成本,且当模型无需详尽表达即可得出正确答案时,效率低下。这促使了将计算转移到隐藏表示中的隐空间推理方法,仅输出最终答案。但许多隐式推理方法依赖于固定的隐式精炼步骤数,需在不同模型和数据集上额外调参以平衡准确率与效率。本文提出 AdaAnchor,一种通过迭代优化附着于输入的隐向量锚点实现静默计算的隐式推理框架。该方法引入自适应终止机制,监控锚点稳定性,在动态收敛时提前停止精炼,从而对简单样本减少步骤,对困难样本保留更多计算资源,统一在最大步数预算下运行。在三个数学应用题基准上的实证评估显示,采用自适应终止的 AdaAnchor 相比固定步数隐式精炼最高提升5%准确率,平均隐式精炼步数减少48-60%。相较于标准推理基线,其生成的 token 数量减少92-93%,将计算过程置于无声的隐空间中,实现了更低输出消耗的精度-效率新权衡。

原文摘要 · Abstract (English)

Token-level Chain-of-Thought (CoT) prompting has become a standard way to elicit multi-step reasoning in large language models (LLMs), especially for mathematical word problems. However, generating long intermediate traces increases output length and inference cost, and can be inefficient when the model could arrive at the correct answer without extensive verbalization. This has motivated latent-space reasoning approaches that shift computation into hidden representations and only emit a final answer. Yet, many latent reasoning methods depend on a fixed number of latent refinement steps at inference, adding another hyperparameter that must be tuned across models and datasets to balance accuracy and efficiency. We introduce AdaAnchor, a latent reasoning framework that performs silent iterative computation by refining a set of latent anchor vectors attached to the input. AdaAnchor further incorporates an adaptive halting mechanism that monitors anchor stability across iterations and terminates refinement once the anchor dynamics converge, allocating fewer steps to easier instances while reserving additional refinement steps for harder ones under a shared maximum-step budget. Our empirical evaluation across three mathematical word-problem benchmarks shows that AdaAnchor with adaptive halting yields accuracy gains of up to 5% over fixed-step latent refinement while reducing average latent refinement steps by 48-60% under the same maximum-step budget. Compared to standard reasoning baselines, AdaAnchor achieves large reductions in generated tokens (92-93%) by moving computation into silent latent refinement, offering a different accuracy-efficiency trade-off with substantially lower output-token usage.

隐空间推理自适应计算大模型优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。