通过模式控制提升测试时计算效率，解决模型重复出错问题。
Mode-Conditioning Unlocks Superior Test-Time Scaling
- 用专家模型或模式前缀分配计算资源，引导多样化推理。
- 在0.5B到7B模型上实现4倍效率提升，最大通过率显著提高。
- 无需标签即可通过梯度聚类实现，适合大规模推理任务。
并行采样可大幅提高测试时计算效率，但易受模式坍缩限制——模型集中于少数模式，导致重复错误。本文提出模式控制（ModC）框架，通过专用模型或模式特定前缀显式分配测试时计算资源。ModC在图搜索任务和大规模推理基准上持续提升性能，覆盖0.5B至7B模型规模。在OpenThoughts上，对Qwen2.5-7B微调后，相比标准训练实现4倍效率提升，且最高可达通过率显著改善。进一步发现，梯度聚类可在无显式模式标签下实现ModC，使NuminaMath等数据集性能提升最高达10%。此外，ModC可增强强化学习，并进一步提升多样性促进型强化学习方法。结果表明，标准训练低估了数据中的多样性，而ModC是一种简单有效的解决方案，能充分释放测试时计算的潜力。
原文摘要 · Abstract (English)
Parallel sampling promises substantial gains in test-time scaling, but its effectiveness is sharply limited by diversity collapse, where models concentrate on a few modes and repeated samples produce the same mistakes. We propose the mode-conditioning (ModC) framework, which explicitly allocates test-time compute across reasoning modes using either specialist models or mode-specific prefixes. ModC consistently improves scaling across controlled graph-search tasks and large-scale reasoning benchmarks, spanning model families and sizes from 0.5B to 7B. On OpenThoughts, fine-tuning Qwen2.5-7B with ModC achieves a 4x efficiency gain over standard training while also improving the maximum attainable Pass@k. We further show that gradient clustering enables ModC without explicit mode labels, yielding up to 10% gains on datasets such as NuminaMath. Finally, we show that ModC improves reinforcement learning (RL) and can further boost diversity-inducing RL methods. These results demonstrate that standard training underutilizes the diversity in data, and that ModC provides a simple, effective remedy for unlocking the full benefits of diversity in test-time scaling.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。