arXiv:2606.02248cs.CL2026-06

用几何路径逼近让大模型推理更短,不靠啰嗦文字也能对。

Geometric Latent Reasoning Induces Shorter Generations in LLMs

论文配图:Geometric Latent Reasoning Induces Shorter Generations in LLMs
图 1 · 摘自论文原文
  • 在嵌入空间中用连续路径替代离散推理步骤
  • 数学推理任务中生成长度减少40%以上,准确率不变
  • 适合追求高效推理的部署场景,尤其对长文本生成有帮助

大型语言模型解决复杂问题依赖冗长的显式推理链,代价高昂且受长度限制。本文将隐式推理建模为预训练词嵌入空间中的几何路径逼近问题,提出几何隐式推理(GLR),通过轻量级过渡头预测嵌入空间中的方向更新。以文本思维链作为锚点,GLR学习近似离散推理轨迹,同时允许对精确词嵌入的连续偏离。在Qwen3模型上进行数学推理基准测试发现:几何隐式推理可显著缩短生成长度,无需显式长度约束即出现此现象。通过用连续隐式步骤替换早期显式推理,模型常以更少总生成步数获得正确答案。结果表明,连续轨迹可作为紧凑的中间推理状态,揭示了隐式计算预算、输出长度与准确率之间的新权衡关系。

原文摘要 · Abstract (English)

Large language models solve complex problems by generating lengthy chains of explicit reasoning tokens. While effective, this makes reasoning expensive, length-sensitive, and constrained to (discrete) natural language. While latent reasoning offers a continuous alternative, determining useful structures for intermediate latent states is an open challenge. In this paper, we formulate latent reasoning as a geometric path-approximation problem within the model's pretrained token-embedding space. We introduce Geometric Latent Reasoning (GLR), which uses a lightweight transition head to predict iterative direction updates in embedding space. Using textual chain-of-thought traces as anchors, GLR learns to approximate discrete reasoning trajectories while permitting continuous deviations from exact token embeddings. Evaluations on mathematical reasoning benchmarks using Qwen3 models reveal an emergent phenomenon: geometric latent reasoning induces substantially shorter generations without an explicit length objective. By replacing early explicit reasoning with continuous latent steps, models often reach correct answers using substantially fewer total generation steps. These findings suggest that continuous trajectories act as compact intermediate reasoning states, exposing a new tradeoff between latent computation budget, output length, and accuracy.

隐式推理生成优化几何建模大模型效率

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。