用世界模型生成交通视频，靠对齐分布而非增大模型提升效果
CosmosAlign: Adapting a World Foundation Model for Generative Traffic Video Forecasting
- 分两阶段用LoRA适配，对齐条件分布与任务需求
- 通过LLM重写描述词，使模型更懂提示词结构
- 推理时无训练融合静态区域运动，提升画面连贯性
生成式交通视频预测旨在从短时观测历史和文本描述中生成长期、时间一致的交通场景未来视频。本文提出CosmosAlign框架，基于预训练的Cosmos3-Nano世界基础模型构建。我们发现，将大型预训练世界模型成功适配到下游预测任务的关键在于分布对齐，而非增加模型容量。为此，我们提出一种两阶段LoRA适配策略：首先对齐条件模式分布与目标预测任务；其次通过基于大语言模型的重描述管道，将训练描述词对齐至模型原生结构化提示接口。推理时，进一步采用完全无需训练的方法，包括基于共识的中位样本选择和动态调整静态场景区域的融合策略，以提升预测质量。CosmosAlign在AI City Challenge 2026 Track 5基准上取得76.49分，位居最终排行榜第一。代码已公开于https://quangminhdinh.github.io/CosmosAlign/。
原文摘要 · Abstract (English)
Generative traffic video forecasting aims to synthesize long-horizon, temporally coherent future videos of traffic scenes from a short observation history and textual descriptions. In this paper, we present CosmosAlign, a generative traffic video forecasting framework built upon the pretrained Cosmos3-Nano world foundation model. Our approach is motivated by the observation that successfully adapting large pretrained world models to downstream forecasting tasks depends primarily on distribution alignment rather than increased model capacity. To this end, we propose a two-stage LoRA adaptation strategy that first aligns the conditioning-mode distribution with the target forecasting task, and then aligns the training captions with the model's native structured prompting interface through an LLM-based re-captioning pipeline. During inference, we further improve prediction quality using a fully training-free procedure consisting of consensus-based medoid sample selection and motion-adaptive blending of static scene regions. CosmosAlign achieves a final score of 76.49 on the AI City Challenge 2026 Track 5 benchmark, ranking first on the final leaderboard. Our code is publicly available at https://quangminhdinh.github.io/CosmosAlign/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。