研究自动驾驶中运动预测与规划模型的缩放规律,发现算力投入越多性能提升越明显。
Scaling Laws of Motion Forecasting and Planning -- Technical Report
- 用50万小时驾驶数据训练自回归Transformer模型,验证性能随算力呈幂律增长。
- 闭环指标随模型规模提升,说明开环评估可能误导模型优化方向。
- 小模型通过采样聚类可媲美大模型,但存在效率拐点,适合资源受限场景。
我们在自动驾驶领域的联合运动预测与规划任务中,研究了一类编码器-解码器自回归Transformer模型的实证缩放规律。基于50万小时驾驶数据,发现模型性能随总计算量呈幂律提升,且训练损失与评估指标高度相关。最值得关注的是,闭环指标也随模型规模增长,这对开环指标在模型迭代中的有效性提出挑战。我们还研究了训练算力最优下的参数量与数据量的最优缩放比例:当训练算力增加时,模型大小应以1.5倍于数据集增长的速度扩展。此外,推理阶段的算力缩放显示,对小模型输出进行采样和聚类可使其性能接近大模型,直至某个交叉点后大模型更具推理效率。整体结果表明,优化训练与推理阶段的缩放策略是提升模型应对复杂驾驶场景能力的关键。最后,我们初步探讨了使用其他智能体的通用驾驶日志数据来提升自身代理性能的可行性,为解决机器人数据稀缺问题提供思路。
原文摘要 · Abstract (English)
We study the empirical scaling laws of a family of encoder-decoder autoregressive transformer models on the task of joint motion forecasting and planning in the autonomous driving domain. Using a 500 thousand hours driving dataset, we demonstrate that, similar to language modeling, model performance improves as a power-law function of the total compute budget, and we observe a strong correlation between model training loss and model evaluation metrics. Most interestingly, closed-loop metrics also improve with scaling, which has important implications for the suitability of open-loop metrics for model development and hill climbing. We also study the optimal scaling of the number of transformer parameters and the training data size for a training compute-optimal model. We find that as the training compute budget grows, optimal scaling requires increasing the model size 1.5x as fast as the dataset size. We also study inference-time compute scaling, where we observe that sampling and clustering the output of smaller models makes them competitive with larger models, up to a crossover point beyond which a larger models becomes more inference-compute efficient. Overall, our experimental results demonstrate that optimizing the training and inference-time scaling properties of motion forecasting and planning models is a key lever for improving their performance to address a wide variety of driving scenarios. Finally, we briefly study the utility of training on general logged driving data of other agents to improve the performance of the ego-agent, an important research area to address the scarcity of robotics data for large capacity models training.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。