arXiv:2603.06222cs.CL2026-03KDD被引 2

让大模型推理更高效可解释,用潜空间停顿代替冗长思维链。

SPOT: Span-level Pause-of-Thought for Efficient and Interpretable Latent Reasoning in Large Language Models

  • 通过段级语义对齐,将思维链压缩为紧凑的潜空间停顿。
  • 平均准确率提升2.3点,生成token减少37.5%。
  • 潜状态可直接解码为关键词,便于理解模型思考过程。

显式思维链虽能提升大模型推理性能,但常因冗长的逐标记追踪导致高推理开销。现有方法通过简短提示或步骤剪枝降低负担,却多是截断输出而非内化思考。潜空间推理提供替代方案,但先前方法面临两大挑战:多数依赖刚性点对点对齐,迫使每个潜标记逼近推理步骤的最终表示,难以捕捉完整推理段落的密集、变长语义;且潜状态缺乏可解释性,通常由无约束优化或嵌入混合生成,难以在预训练语言头下解码审计。本文提出SPOT框架,将显式思维链压缩为无需固定响应模板的紧凑潜空间停顿。核心是段级语义对齐,基于Sinkhorn最优传输目标,软性匹配每个停顿标记与整个推理段落的语义,突破步骤终点对齐的僵化限制。为进一步提升可解释性,引入冻结头解码约束,确保潜状态可直接在冻结的预训练语言头下解码为标记分布,实现对潜思考过程的可读关键词解释。在多个推理基准测试中,SPOT平均准确率提升2.3点,生成标记数减少37.5%,并提供忠实的语义解释。

原文摘要 · Abstract (English)

Explicit Chain-of-Thought improves the reasoning performance of large language models but often incurs high inference cost due to verbose token-level traces. While recent approaches reduce this overhead via concise prompting or step pruning, they largely truncate what the model says rather than internalize what the model thinks. Latent reasoning offers a promising alternative by performing computation in the hidden space, yet prior methods face two critical challenges. Many existing approaches rely on rigid point-to-point alignment, forcing a latent token to approximate the final representation of a reasoning step, which can be insufficient to capture the dense, variable-length semantics of an entire reasoning segment. Furthermore, these methods often suffer from a lack of interpretability: latent states are commonly produced by unconstrained optimization or embedding mixing, yielding vectors that are difficult to decode or audit under the pretrained language head. We propose SPOT, a flexible framework that compresses explicit CoT into compact latent pause tokens without enforcing a fixed response template. At the core of SPOT is Span-level Semantic Alignment, a Sinkhorn optimal-transport objective that softly matches each pause token to the semantics of an entire reasoning segment, overcoming the rigidity of step-end alignment. To further improve interpretability, SPOT introduces a Frozen-Head Decoding Constraint that keeps latent states directly decodable as token distributions under the frozen pretrained LM head, enabling readable keyword interpretations of latent thoughts. Experiments on reasoning benchmarks demonstrate that SPOT improves accuracy by 2.3 points on average while reducing generated tokens by 37.5% and provides faithful semantic interpretations of the latent reasoning process.

大模型推理潜空间可解释性思维链

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。