通过隐空间优化推理路径，提升大模型数学推理准确率。
Efficient Post-Training Refinement of Latent Reasoning in Large Language Models
- 在模型隐空间内对比强弱推理路径，引导方向性更新
- 5个基准测试中实现显著提效，MathQA提升5%准确率
- 适合需要轻量级推理优化的场景，如低资源部署
推理是大语言模型语言理解的核心。尽管思维链提示能通过显式中间步骤提升性能，但存在冗余令牌开销和固定推理轨迹的问题，难以进行分步优化。近期隐空间推理方法通过直接在模型隐状态中优化内部推理过程，避免生成显式输出，解决了上述问题。然而关键挑战仍在于：如何在后训练阶段有效更新推理嵌入以引导模型趋向更准确解。为此，本文提出一种轻量级后训练框架，采用两种新策略：1）对比推理反馈，通过对比强/弱基线推理嵌入，推断有效更新方向并增强嵌入；2）残差嵌入优化，通过逐步融合当前与历史梯度，稳定更新过程，实现快速且可控的收敛。在五个推理基准上开展大量实验与案例研究，验证了该框架的有效性。特别地，在MathQA上实现了5%的准确率提升，无需额外训练。
原文摘要 · Abstract (English)
Reasoning is a key component of language understanding in Large Language Models. While Chain-of-Thought prompting enhances performance via explicit intermediate steps, it suffers from sufficient token overhead and a fixed reasoning trajectory, preventing step-wise refinement. Recent advances in latent reasoning address these limitations by refining internal reasoning processes directly in the model's latent space, without producing explicit outputs. However, a key challenge remains: how to effectively update reasoning embeddings during post-training to guide the model toward more accurate solutions. To overcome this challenge, we propose a lightweight post-training framework that refines latent reasoning trajectories using two novel strategies: 1) Contrastive reasoning feedback, which compares reasoning embeddings against strong and weak baselines to infer effective update directions via embedding enhancement; 2) Residual embedding refinement, which stabilizes updates by progressively integrating current and historical gradients, enabling fast yet controlled convergence. Extensive experiments and case studies are conducted on five reasoning benchmarks to demonstrate the effectiveness of the proposed framework. Notably, a 5\% accuracy gain on MathQA without additional training.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。