arXiv:2507.14419cs.LGcs.AI2025-07被引 4

简单测试时缩放本质是限长降性能,非真正提升能力。

It's Not That Simple. An Analysis of Simple Test-Time Scaling

  • 通过限制生成长度模拟缩放行为,而非真正增加计算量。
  • 用'Wait'追加法会导致模型解题来回震荡,结果不稳定。
  • 真正高效缩放需模型自主学习计算资源分配,适合研究推理优化者。

先前工作提出简单测试时缩放方法,通过从o1类模型中蒸馏出的模型,手动控制测试时计算量:或通过强制最大长度来缩放降低,或通过在模型即将终止时反复追加'Wait'来缩放增加。本文分析发现,该缩放行为主要源于强制最大长度的缩放下降。相反,在从o1类模型蒸馏的长思维链数据上微调,对缩放行为无显著影响;而追加'Wait'的方式导致不一致性,模型可能在不同解之间振荡。关键区别在于:o1类模型(如DeepSeek-R1@)通常可自由使用计算资源,唯一约束是最大支持长度。通过强化学习训练,这些模型能自然地在测试时扩展计算量,从而超越原始峰值性能。而简单测试时缩放在缩放下降时逐步压低性能上限。尽管可通过缩放下降复现o1模型的缩放现象,但其真正目标是突破模型原有限制以获得更高性能,而非仅模仿缩放表象。

原文摘要 · Abstract (English)

Prior work proposed simple test-time scaling, a method for replicating this scaling behavior with models distilled from o1-like models by manually controlling test-time compute: either scaling down by enforcing a maximum length or scaling up by iteratively appending "Wait" when the model is about to terminate its generation. This paper presents an analysis of simple test-time scaling and finds that the scaling behavior is largely attributed to scaling down by enforcing a maximum length. In contrast, fine-tuning on long CoT data distilled from o1-like models has no significant impact on scaling behavior, and scaling up by appending "Wait" leads to inconsistencies, as the model may oscillate between solutions. A key distinction exists between scaling down by enforcing a maximum length and scaling up test-time compute in o1-like models, such as DeepSeek-R1\@. These models are typically allowed to utilize as much compute as needed, with the only constraint being the model's maximum supported length. By learning to naturally scale up test-time compute during reinforcement learning, o1-like models surpass their peak performance when scaling up. In contrast, simple test-time scaling progressively imposes a lower upper limit on model performance as it scales down. While replicating the test-time scaling behavior of o1 models can be straightforward by scaling down, it is crucial to recognize that the goal of scaling test-time compute is to unlock higher performance -- beyond what the model could originally achieve -- rather than merely reproducing the appearance of scaling behavior.

测试时缩放推理优化模型性能

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。