arXiv:2505.11484cs.CL2025-05被引 26

通过连续思维空间的多样化探索,提升大模型推理能力。

SoftCoT++: Test-Time Scaling with Soft Chain-of-Thought Reasoning

  • 在连续潜空间中引入多初始扰动,实现思维路径多样化。
  • 在5个基准上均超越SoftCoT,且优于自一致性扩展方法。
  • 兼容传统缩放技术,适合需要高精度推理的应用场景。

测试时扩展(Test-Time Scaling, TTS)指在不修改模型参数的前提下,通过增加推理阶段计算量来提升推理性能的方法。现有TTS方法在离散词元空间中生成更多中间步骤,而近期研究Coconut和SoftCoT表明,在连续潜空间中进行思考能进一步提升推理表现。此类潜思无信息损失,但单次输入对应的潜表示固定,导致推理路径缺乏多样性。为此,本文提出SoftCoT++,将SoftCoT拓展至测试时扩展范式,通过多个专用初始词元对潜思考进行扰动,并采用对比学习促进潜思表示的多样性。在五个推理基准和两种不同LLM架构上的实验表明,SoftCoT++显著优于SoftCoT,且超过带有自一致性缩放的SoftCoT。同时,其与自一致性等传统缩放技术具有良好兼容性。代码已开源:https://github.com/xuyige/SoftCoT。

原文摘要 · Abstract (English)

Test-Time Scaling (TTS) refers to approaches that improve reasoning performance by allocating extra computation during inference, without altering the model's parameters. While existing TTS methods operate in a discrete token space by generating more intermediate steps, recent studies in Coconut and SoftCoT have demonstrated that thinking in the continuous latent space can further enhance the reasoning performance. Such latent thoughts encode informative thinking without the information loss associated with autoregressive token generation, sparking increased interest in continuous-space reasoning. Unlike discrete decoding, where repeated sampling enables exploring diverse reasoning paths, latent representations in continuous space are fixed for a given input, which limits diverse exploration, as all decoded paths originate from the same latent thought. To overcome this limitation, we introduce SoftCoT++ to extend SoftCoT to the Test-Time Scaling paradigm by enabling diverse exploration of thinking paths. Specifically, we perturb latent thoughts via multiple specialized initial tokens and apply contrastive learning to promote diversity among soft thought representations. Experiments across five reasoning benchmarks and two distinct LLM architectures demonstrate that SoftCoT++ significantly boosts SoftCoT and also outperforms SoftCoT with self-consistency scaling. Moreover, it shows strong compatibility with conventional scaling techniques such as self-consistency. Source code is available at https://github.com/xuyige/SoftCoT.

推理增强连续思维测试时扩展

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。