arXiv:2605.07315cs.CL2026-05

用隐空间探索+显式验证,让大模型推理更省 token 又更准

LaTER: Efficient Test-Time Reasoning via Latent Exploration and Explicit Verification

论文配图:LaTER: Efficient Test-Time Reasoning via Latent Exploration and Explicit Verification
图 1 · 摘自论文原文
  • 先在连续隐空间探索,再转为显式推理验证答案
  • 零训练版本减少16%-32% token,AIME准确率提至73.3%
  • 可训练版达80.0%准确率,比标准CoT高10分且少用33% token

链式思维(CoT)提升大模型解决难题能力,但每次推理需生成离散中间步骤,导致推理成本高。隐空间推理通过传播连续状态减少可见词元生成,但可能削弱需符号验证的任务表现。我们提出隐空间-显式推理(LaTER),分两阶段:先在连续隐空间内进行受限探索,再切换到显式CoT进行验证与答案生成。无训练版本中,将最终层隐藏状态投影回输入嵌入空间,保留隐式键值缓存,并利用熵值和模型原生停止标记探测决定切换时机。发现强推理模型在此接口下已呈现结构化隐轨迹。在Qwen3-14B上,训练自由的LaTER在多个基准上减少16%-32%总词元使用量,多数任务准确率持平或提升;例如,在AIME 2025上准确率从70.0%升至73.3%,词元数由15,730降至10,661。我们进一步构建了包含浓缩解题直觉与缩短显式推导的监督语料Latent-Switch-69K。通过隐式推进与终止监督微调,训练版LaTER在AIME 2025上达到80.0%准确率,较标准CoT基线提升10.0分,同时词元减少33%。代码、数据与模型已在https://github.com/TioeAre/LaTER公开。

原文摘要 · Abstract (English)

Chain-of-thought (CoT) reasoning improves large language models (LLMs) on difficult tasks, but it also makes inference expensive because every intermediate step must be generated as a discrete token. Latent reasoning reduces visible token generation by propagating continuous states, yet replacing explicit derivations with latent computation can hurt tasks that require symbolic checking. We propose Latent-Then-Explicit Reasoning (LaTER), a two-stage paradigm that first performs bounded exploration in a continuous latent space and then switches to explicit CoT for verification and answer generation. In a training-free instantiation, LaTER projects final-layer hidden states back to the input embedding space, preserves the latent KV cache, and uses entropy and model-native stop-token probes to decide when to switch. We find that strong reasoning models already exhibit structured latent trajectories under this interface. On Qwen3-14B, training-free LaTER reduces total token usage by 16%-32% on several benchmarks while matching or improving accuracy on most of them; for example, it improves AIME 2025 from 70.0% to 73.3% while reducing tokens from 15,730 to 10,661. We further construct Latent-Switch-69K, a supervised corpus that pairs condensed solution intuitions with shortened explicit derivations. Fine-tuning with latent rollout and halting supervision yields additional gains: trained LaTER reaches 80.0% accuracy on AIME 2025, 10.0 points above the standard CoT baseline, while using 33% fewer tokens. Our code, data, and model are available at https://github.com/TioeAre/LaTER.

大模型推理隐空间高效推理CoT优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。