arXiv:2602.14077cs.CLcs.LG2026-02

提出可学习的高斯思维采样器,让推理过程更可控、更高效。

GTS: Inference-Time Scaling of Latent Reasoning with a Learnable Gaussian Thought Sampler

  • 用可学习的条件分布替代随机扰动,实现可控的潜空间探索。
  • 在多个基准上优于传统启发式方法,提升推理路径质量与决策准确率。
  • 适合追求推理稳定性和可优化性的模型开发者使用。

在潜空间推理模型中,推理时扩展(ITS)通常依赖于启发式扰动,如丢弃或固定高斯噪声,以生成多样化的候选推理轨迹。然而我们发现,更强的扰动并不一定带来更好的采样质量:它们往往引起更大的分布偏移,却未能产生更有价值的推理路径或更优的最终决策。关键问题在于这些扰动注入随机性但未定义明确的条件采样分布,导致潜空间探索难以控制或优化。为此,我们提出高斯思维采样器(GTS),一个轻量级模块,将潜空间探索重构为从学习到的连续推理状态条件分布中采样。GTS预测上下文相关的扰动分布,并采用类似GRPO的策略优化进行训练,同时保持主干模型冻结,将启发式扰动转化为显式的概率采样策略。在多个基准和两种潜空间推理架构上的实验表明,GTS在推理时扩展方面显著优于启发式基线,表明有效的潜空间ITS需要更受控且可优化的采样机制,而非简单放大随机性。

原文摘要 · Abstract (English)

Inference-time scaling (ITS) in latent reasoning models typically relies on heuristic perturbations, such as dropout or fixed Gaussian noise, to generate diverse candidate trajectories. However, we show that stronger perturbations do not necessarily yield better sampling quality: they often induce larger distribution shifts without producing more useful reasoning paths or better final decisions. A key limitation is that these perturbations inject stochasticity without defining an explicit conditional sampling distribution, making latent exploration difficult to control or optimize. To address this, we propose the Gaussian Thought Sampler (GTS), a lightweight module that reformulates latent exploration as sampling from a learned conditional distribution over continuous reasoning states. GTS predicts context-dependent perturbation distributions and is trained with GRPO-style policy optimization while keeping the backbone frozen, turning heuristic perturbation into an explicit probabilistic sampling policy. Experiments across multiple benchmarks and two latent reasoning architectures show that GTS yields more reliable inference-time scaling than heuristic baselines, suggesting that effective latent ITS requires better-controlled and optimizable sampling rather than simply amplifying stochasticity.

推理优化潜空间采样策略

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。