用仿真生成多视角一致的路侧感知数据，提升车辆环境感知能力
RoCo-Sim: Enhancing Roadside Collaborative Perception through Foreground Simulation
- 通过动态前景编辑和全场景风格迁移，生成多视角一致的仿真数据
- 在Rcooper-Intersection和TUMTraf-V2X上，AP70指标分别提升83.74和83.12
- 适合做路侧协同感知、自动驾驶数据增强的研究者使用
路侧协同感知系统通过多个路侧单元协作整合感知数据，提升车辆对环境的认知。现有方法聚焦模型设计，忽视校准误差、信息稀疏和多视角一致性等数据问题，导致在最新公开数据集上表现不佳。为此，我们提出首个面向路侧协同感知的仿真框架RoCo-Sim。该框架通过单图的动态前景编辑与全场景风格迁移，生成多样且多视角一致的仿真路侧数据。RoCo-Sim包含四个组件：(1) 相机外参优化，确保路侧相机的3D到2D投影准确；(2) 新型多视角遮挡感知采样器（MOAS），决定数字资产在3D空间中的放置位置；(3) DepthSAM，从单帧固定视角图像中建模前景-背景关系，保障前景多视角一致性；(4) 可扩展后处理工具包，通过风格迁移等手段生成更真实丰富的场景。RoCo-Sim显著提升路侧3D目标检测性能，在Rcooper-Intersection和TUMTraf-V2X上，AP70指标分别超越最先进方法83.74和83.12。该工作填补了路侧感知仿真领域的关键空白。代码与预训练模型即将开源：https://github.com/duyuwen-duen/RoCo-Sim
原文摘要 · Abstract (English)
Roadside Collaborative Perception refers to a system where multiple roadside units collaborate to pool their perceptual data, assisting vehicles in enhancing their environmental awareness. Existing roadside perception methods concentrate on model design but overlook data issues like calibration errors, sparse information, and multi-view consistency, leading to poor performance on recent published datasets. To significantly enhance roadside collaborative perception and address critical data issues, we present the first simulation framework RoCo-Sim for road-side collaborative perception. RoCo-Sim is capable of generating diverse, multi-view consistent simulated roadside data through dynamic foreground editing and full-scene style transfer of a single image. RoCo-Sim consists of four components: (1) Camera Extrinsic Optimization ensures accurate 3D to 2D projection for roadside cameras; (2) A novel Multi-View Occlusion-Aware Sampler (MOAS) determines the placement of diverse digital assets within 3D space; (3) DepthSAM innovatively models foreground-background relationships from single-frame fixed-view images, ensuring multi-view consistency of foreground; and (4) Scalable Post-Processing Toolkit generates more realistic and enriched scenes through style transfer and other enhancements. RoCo-Sim significantly improves roadside 3D object detection, outperforming SOTA methods by 83.74 on Rcooper-Intersection and 83.12 on TUMTraf-V2X for AP70. RoCo-Sim fills a critical gap in roadside perception simulation. Code and pre-trained models will be released soon: https://github.com/duyuwen-duen/RoCo-Sim
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。