Glad实现自动驾驶场景的逐帧视频生成,支持长时序模拟。
Glad: A Streaming Scene Generator for Autonomous Driving
- 逐帧生成视频,用潜变量传播保证时间一致性
- 在nuScenes数据集上生成视频长度超30秒,效果优于现有方法
- 适合用于自动驾驶的极端场景仿真与在线数据生成
真实世界场景的生成与仿真在自动驾驶领域具有重要应用价值,尤其在边缘案例测试中。近年来,研究者尝试使用神经辐射场或扩散模型生成驾驶场景的新视角或合成数据,但这些方法存在无法生成未见场景或视频长度受限的问题,难以满足数据生成与仿真的适应性需求。为此,我们提出一种简单而有效的框架Glad,以逐帧方式生成视频数据。为确保合成视频的时间一致性,引入潜变量传播模块,将前一帧的潜特征作为噪声先验注入当前帧的潜特征中。此外,设计流式数据采样器,连续迭代地有序采样视频片段中的原始图像。给定参考帧后,Glad可视为一个流式模拟器,生成特定场景的视频。在广泛使用的nuScenes数据集上进行了大量实验,结果表明,Glad实现了优异性能,可作为在线视频生成的强基线。源代码和模型将公开发布。
原文摘要 · Abstract (English)
The generation and simulation of diverse real-world scenes have significant application value in the field of autonomous driving, especially for the corner cases. Recently, researchers have explored employing neural radiance fields or diffusion models to generate novel views or synthetic data under driving scenes. However, these approaches suffer from unseen scenes or restricted video length, thus lacking sufficient adaptability for data generation and simulation. To address these issues, we propose a simple yet effective framework, named Glad, to generate video data in a frame-by-frame style. To ensure the temporal consistency of synthetic video, we introduce a latent variable propagation module, which views the latent features of previous frame as noise prior and injects it into the latent features of current frame. In addition, we design a streaming data sampler to orderly sample the original image in a video clip at continuous iterations. Given the reference frame, our Glad can be viewed as a streaming simulator by generating the videos for specific scenes. Extensive experiments are performed on the widely-used nuScenes dataset. Experimental results demonstrate that our proposed Glad achieves promising performance, serving as a strong baseline for online video generation. We will release the source code and models publicly.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。