90亿参数模型在5500小时驾驶数据上训练,刷新自动驾驶视频生成开源纪录。
How Far Can 5,500 Hours of Driving Take You? A Scaling Law Analysis of Video Diffusion Models

- 基于100万到90亿参数模型,研究模型规模与训练时长的扩展规律。
- 训练时长对性能提升效果远超模型大小,但大模型最终损失更低。
- 适用于追求高精度自动驾驶视频生成的研究者与开发者。
自动驾驶视频生成无法走网页规模路线:驾驶数据收集成本高,受隐私限制,不能随意采集,因此模型必须充分利用有限数据。本文系统研究了从零训练的视频扩散模型在驾驶数据上的扩展规律:构建了从100万到90亿参数的一系列模型,训练时长最高达5,500小时。验证损失在模型规模和训练暴露量上均呈现一致的幂律关系,回答了训练预算的关键问题:是应投入更多算力延长训练时间,还是扩大模型规模?结果表明,训练时长带来的性能提升远快于模型规模增长,因此在算力有限时,更长训练更有效;但更大模型仍能实现更低的渐近损失,当算力与数据充足时,最优策略仍是增大模型规模。基于此规律,我们训练了一个90亿参数模型,据作者所知,这是首个从零开始在驾驶数据上训练的最大视频扩散模型,其在nuScenes数据集上达到新的开源最先进水平。代码与预训练模型已开源:https://github.com/valeoai/VATIX。NATIX将分阶段发布底层驾驶数据。
原文摘要 · Abstract (English)
Video generation for autonomous driving cannot follow the web-scale route: driving data is expensive to collect, bound by privacy requirements, and cannot be scraped at will, so models must make the most of a fixed corpus. We present a systematic scaling-law study of video diffusion models trained from scratch on driving data: a family of models from 1M to 9B parameters, trained at different exposures on up to 5,500 hours of driving. Validation loss follows consistent power laws in both model size and training exposure, answering the questions that shape a training budget: whether compute is better spent on longer training or on a larger model, and whether more data is needed. Loss improves much faster with training exposure than with model size, making longer training the most effective way to improve a fixed model under limited compute. However, larger models continue to achieve lower asymptotic loss, so compute-optimal scaling still favors increasing model size when sufficient compute and data are available. Guided by these laws, we train a 9B-parameter model, to our knowledge the largest video diffusion model trained from scratch on driving data: it sets a new open-source state of the art for driving video generation, as measured on nuScenes. Our code and pretrained models are available at https://github.com/valeoai/VATIX. NATIX is separately releasing the underlying driving data in stages.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。