arXiv:2509.15479cs.CV2025-09

用开源模型+公开数据,实现低延迟汽车驾驶视频生成。

OpenViGA: Video Generation for Automotive Driving Scenes by Streamlining and Fine-Tuning Open Source Models with Public Data

  • 基于开源模型微调,分模块评估图像编码、世界模型和解码器。
  • 256x256分辨率下4帧/秒,单帧延迟完成生成。
  • 代码数据全公开,支持复现,适合自动驾驶研究者使用。

近期成功的驾驶场景视频生成系统将图像分词、未来状态预测(世界模型)和视频解码分配给专用模型,通常依赖大型模型且需大量训练资源,设计透明度低,代码与数据不公开。本文提出OpenViGA,一个面向汽车驾驶场景的开源视频生成系统。我们首次对系统的三大组件(图像分词器、世界模型、视频解码器)进行独立定量与定性评估。系统完全基于跨领域预训练开源模型,仅在学术级GPU上利用公开汽车数据集BDD100K进行微调。通过简化各组件接口,构建统一生成流程。由于底层模型与数据均公开,系统具备完全可复现性,并已发布于GitHub。在256x256分辨率、4帧/秒条件下,系统可实现逐帧生成,算法延迟仅为单帧。

原文摘要 · Abstract (English)

Recent successful video generation systems that predict and create realistic automotive driving scenes from short video inputs assign tokenization, future state prediction (world model), and video decoding to dedicated models. These approaches often utilize large models that require significant training resources, offer limited insight into design choices, and lack publicly available code and datasets. In this work, we address these deficiencies and present OpenViGA, an open video generation system for automotive driving scenes. Our contributions are: Unlike several earlier works for video generation, such as GAIA-1, we provide a deep analysis of the three components of our system by separate quantitative and qualitative evaluation: Image tokenizer, world model, video decoder. Second, we purely build upon powerful pre-trained open source models from various domains, which we fine-tune by publicly available automotive data (BDD100K) on GPU hardware at academic scale. Third, we build a coherent video generation system by streamlining interfaces of our components. Fourth, due to public availability of the underlying models and data, we allow full reproducibility. Finally, we also publish our code and models on Github. For an image size of 256x256 at 4 fps we are able to predict realistic driving scene videos frame-by-frame with only one frame of algorithmic latency.

视频生成自动驾驶开源模型扩散模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。