通过迭代优化隐藏状态，提升大模型推理的稳定性和准确性。
Learning to Refine Hidden States for Reliable LLM Reasoning
- 用强化学习动态调整隐藏状态的迭代次数和方向。
- 在多个基准上准确率提升，推理过程更稳定。
- 无需显式思维链，推理开销更低，适合实际部署。
大语言模型虽具备强大推理能力，但在复杂多步任务中内部推理过程易受早期隐藏状态误差影响，导致错误传播。本文提出ReLAR，一种基于强化学习的潜在状态精炼框架，在解码前迭代更新隐藏表示。ReLAR维护紧凑的潜在推理状态，通过学习到的深度与动作控制器自适应决定精炼步骤的数量与方向。控制器基于每步似然提升的策略梯度目标训练，实现高效、依赖输入的推理，无需显式生成思维链。在医学、数学、多跳推理和开放式生成等多个基准上的实验表明，ReLAR显著提升准确率、生成质量与推理稳定性，且推理开销远低于显式推理基线。
原文摘要 · Abstract (English)
Large language models show strong reasoning ability, but their internal reasoning process can remain unstable in complex multi-step settings, where early hidden-state errors may propagate to incorrect predictions. We propose ReLAR, a reinforcement-guided latent refinement framework that iteratively updates hidden representations before decoding. ReLAR maintains a compact latent reasoning state and uses learned depth and action controllers to adaptively determine both the number and direction of refinement steps. The controllers are trained with a policy gradient objective based on step-wise likelihood improvement, enabling efficient input-dependent reasoning without explicit chain-of-thought generation. Experiments on medical, mathematical, multi-hop reasoning, and open-ended generation benchmarks show that ReLAR improves accuracy, generation quality, and reasoning stability with substantially lower inference overhead than explicit reasoning baselines.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。