解决自动驾驶视频生成中物体身份漂移问题，保持物体外观一致
ConsisDrive: Identity-Preserving Driving World Models for Video Generation by Instance Mask
- 用实例掩码注意力机制约束物体跨帧特征交互
- 在nuScenes上生成视频质量达顶尖水平，下游任务性能提升
- 适合需要真实、稳定物体外观的自动驾驶数据生成场景
自动驾驶依赖大规模高质量多视角驾驶视频训练鲁棒模型。尽管世界模型能低成本生成逼真驾驶数据，但常因缺乏实例级时序约束而出现身份漂移——同一物体在不同帧中外观或类别发生变化。我们提出ConsisDrive，一种保持物体身份一致性的驾驶世界模型。框架包含两个关键组件：(1) 实例掩码注意力，在注意力模块中引入实例身份掩码与轨迹掩码，确保视觉令牌仅与对应实例特征在时空维度交互，从而维持物体身份一致性；(2) 实例掩码损失，通过概率性实例掩码自适应增强前景区域，减少背景噪声同时保持整体场景保真度。集成上述机制后，ConsisDrive在nuScenes数据集上实现了领先的驾驶视频生成质量，并在下游自动驾驶任务中显著提升性能。
原文摘要 · Abstract (English)
Autonomous driving relies on robust models trained on large-scale, high-quality multi-view driving videos. Although world models provide a cost-effective solution for generating realistic driving data, they often suffer from identity drift, where the same object changes its appearance or category across frames due to the absence of instance-level temporal constraints. We introduce ConsisDrive, an identity-preserving driving world model designed to enforce temporal consistency at the instance level. Our framework incorporates two key components: (1) Instance-Masked Attention, which applies instance identity masks and trajectory masks within attention blocks to ensure that visual tokens interact only with their corresponding instance features across spatial and temporal dimensions, thereby preserving object identity consistency; and (2) Instance-Masked Loss, which adaptively emphasizes foreground regions with probabilistic instance masking, reducing background noise while maintaining overall scene fidelity. By integrating these mechanisms, ConsisDrive achieves state-of-the-art driving video generation quality and demonstrates significant improvements in downstream autonomous driving tasks on the nuScenes dataset. Our project page is https://shanpoyang654.github.io/ConsisDrive/page.html.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。