测试时扩展让世界模型推理更高效,无需重训或增大模型
Can Test-Time Scaling Improve World Foundation Model?
- 设计可扩展的测试时推理框架,结合快速分词与剪枝策略
- 在COSMOS模型上验证测试时缩放定律成立,提升推理效率
- 适合需要高效推理的自动驾驶与机器人场景
世界基础模型(WFM)通过从当前观测和输入预测未来状态,已成为物理智能应用的核心,如自动驾驶和机器人。然而,这些模型预训练需大量计算资源,且后训练阶段受数据量限制。因此,测试时扩展成为替代传统模型放大或重训练的关键且可行方案。本文提出SWIFT框架,集成可扩展的WFM评估工具与过程级推理策略,包括快速分词、基于概率的Top-K剪枝和高效束搜索。在COSMOS模型上的实证结果表明,即使在计算最优条件下,测试时扩展依然有效。研究发现,测试时缩放规律适用于WFM,SWIFT为提升推理性能提供了无需重训或增大模型的可扩展路径。
原文摘要 · Abstract (English)
World foundation models, which simulate the physical world by predicting future states from current observations and inputs, have become central to many applications in physical intelligence, including autonomous driving and robotics. However, these models require substantial computational resources for pretraining and are further constrained by available data during post-training. As such, scaling computation at test time emerges as both a critical and practical alternative to traditional model enlargement or re-training. In this work, we introduce SWIFT, a test-time scaling framework tailored for WFMs. SWIFT integrates our extensible WFM evaluation toolkit with process-level inference strategies, including fast tokenization, probability-based Top-K pruning, and efficient beam search. Empirical results on the COSMOS model demonstrate that test-time scaling exists even in a compute-optimal way. Our findings reveal that test-time scaling laws hold for WFMs and that SWIFT provides a scalable and effective pathway for improving WFM inference without retraining or increasing model size. Project page: https://scalingwfm.github.io/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。