用世界模型生成逼真自动驾驶场景数据,提升感知模型性能。
SimWorld: A Unified Benchmark for Simulator-Conditioned Scene Generation via World Model
- 结合仿真引擎与世界模型,实现受控场景生成
- 生成图像显著提升下游感知模型准确率
- 适合自动驾驶数据增强与真实场景模拟研究
随着自动驾驶技术快速发展,数据不足已成为制约感知模型精度提升的主要障碍。当前研究正探索利用世界模型实现可控数据生成以丰富数据集。然而,以往工作多聚焦于特定公开数据集上的图像生成质量,缺乏对真实应用场景下大规模复杂场景数据生成引擎的研究。本文提出一种基于世界模型的仿真条件化场景生成引擎,通过构建与真实场景一致的仿真系统,可采集任意场景的仿真数据及标签,作为世界模型生成的条件。该方法融合仿真引擎的强大场景生成能力与世界模型的鲁棒生成能力,形成全新数据生成流程。此外,本文构建了一个虚拟与真实数据比例均衡的基准测试集,用于评估世界模型在真实场景中的生成能力。定量结果表明,生成图像能显著提升下游感知模型性能。最后,我们针对城市自动驾驶场景进行了生成能力探索。所有数据与代码将开源于 https://github.com/Li-Zn-H/SimWorld。
原文摘要 · Abstract (English)
With the rapid advancement of autonomous driving technology, a lack of data has become a major obstacle to enhancing perception model accuracy. Researchers are now exploring controllable data generation using world models to diversify datasets. However, previous work has been limited to studying image generation quality on specific public datasets. There is still relatively little research on how to build data generation engines for real-world application scenes to achieve large-scale data generation for challenging scenes. In this paper, a simulator-conditioned scene generation engine based on world model is proposed. By constructing a simulation system consistent with real-world scenes, simulation data and labels, which serve as the conditions for data generation in the world model, for any scenes can be collected. It is a novel data generation pipeline by combining the powerful scene simulation capabilities of the simulation engine with the robust data generation capabilities of the world model. In addition, a benchmark with proportionally constructed virtual and real data, is provided for exploring the capabilities of world models in real-world scenes. Quantitative results show that these generated images significantly improve downstream perception models performance. Finally, we explored the generative performance of the world model in urban autonomous driving scenarios. All the data and code will be available at https://github.com/Li-Zn-H/SimWorld.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。