让大模型在连续空间中更稳定地思考，提升推理能力。
Latent Thoughts Tuning: Bridging Context and Reasoning with Fused Information in Latent Tokens
- 用上下文与语义引导融合构建潜在思维
- 减少特征坍缩，推理准确率显著提升
- 支持隐式与显式思考模式动态切换
尽管显式思维链（CoT）赋予大语言模型强大推理能力，但其将思维限制在离散词汇空间。近年来，连续潜在空间推理成为有前景的替代方案，但现有方法因反复使用隐藏状态导致分布不匹配而出现特征坍缩与不稳，或依赖助手模型引发对齐问题。为此，我们提出潜思调优（LT-Tuning），一种后训练框架，重新定义潜在思维的构建与部署方式。该方法不依赖原始隐藏状态，而是引入上下文-预测-融合机制，联合利用上下文隐藏状态与词汇嵌入空间中的预测语义引导。结合渐进式三阶段课程学习策略，LT-Tuning 还实现潜在与显式思维模式的动态切换。实验表明，该方法优于现有潜在推理基线，有效缓解特征坍缩，实现稳健的推理准确率。
原文摘要 · Abstract (English)
While explicit Chain-of-Thought (CoT) equips Large Language Models (LLMs) with strong reasoning capabilities, it constrains the model's thoughts to a discrete vocabulary space. Recently, reasoning in continuous latent space has emerged as a promising alternative, but current paradigms suffer from feature collapse and instability due to distribution mismatch when recurrently reusing hidden states, or alignment issues when relying on assistant models. To address this, we propose Latent Thoughts Tuning (LT-Tuning), a post-training framework that redefines how latent thoughts are constructed and deployed. Instead of relying solely on raw hidden states, our method introduces a Context-Prediction-Fusion mechanism that jointly leverages contextual hidden states and predictive semantic guidance from the vocabulary embedding space. Combined with a progressive three-stage curriculum learning pipeline, LT-Tuning also enables dynamic switching between latent and explicit thinking modes. Experiments demonstrate that our method outperforms existing latent reasoning baselines, effectively mitigating feature collapse and achieving robust reasoning accuracy.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。