用事件相机提升自动驾驶场景补全,增强恶劣条件下的感知鲁棒性。
Event-aided Semantic Scene Completion
- 融合RGB与事件数据,通过动态特征映射实现3D空间重建。
- 在五种退化场景下mIoU提升最高达52.5%,尤其在传感器部分失效时表现优异。
- 首个真实世界事件辅助场景补全基准,适合自动驾驶感知研究者使用。
自动驾驶系统依赖可靠的三维场景理解。现有基于RGB的语义场景补全(SSC)方法在运动模糊、光照不足和恶劣天气下表现受限。事件相机具备高动态范围和低延迟特性,能提供异步数据以补充RGB输入。本文提出首个真实世界事件辅助语义场景补全基准DSEC-SSC,包含新颖的4D标注流程,生成随物体运动动态调整的稠密、可见性感知标签。提出的RGB-事件融合框架EvSSC引入事件辅助提升模块(ELM),有效将2D RGB-事件特征映射至3D空间,增强视角变换能力与3D体构建的鲁棒性。在DSEC-SSC与模拟SemanticKITTI-E上的大量实验表明,EvSSC可适配基于Transformer与LSS的SSC架构。特别地,在SemanticKITTI-C上,其在五种退化模式下均实现稳定提升,图像传感器部分失效时mIoU相对提高52.5%。定量与定性分析验证了其在运动模糊和极端天气条件下的优越性。数据集与代码将公开于https://github.com/Pandapan01/EvSSC。
原文摘要 · Abstract (English)
Autonomous driving systems rely on robust 3D scene understanding. Recent advances in Semantic Scene Completion (SSC) for autonomous driving underscore the limitations of RGB-based approaches, which struggle under motion blur, poor lighting, and adverse weather. Event cameras, offering high dynamic range and low latency, address these challenges by providing asynchronous data that complements RGB inputs. We present DSEC-SSC, the first real-world benchmark specifically designed for event-aided SSC, which includes a novel 4D labeling pipeline for generating dense, visibility-aware labels that adapt dynamically to object motion. Our proposed RGB-Event fusion framework, EvSSC, introduces an Event-aided Lifting Module (ELM) that effectively bridges 2D RGB-Event features to 3D space, enhancing view transformation and the robustness of 3D volume construction across SSC models. Extensive experiments on DSEC-SSC and simulated SemanticKITTI-E demonstrate that EvSSC is adaptable to both transformer-based and LSS-based SSC architectures. Notably, evaluations on SemanticKITTI-C demonstrate that EvSSC achieves consistently improved prediction accuracy across five degradation modes and both In-domain and Out-of-domain settings, achieving up to a 52.5% relative improvement in mIoU when the image sensor partially fails. Additionally, we quantitatively and qualitatively validate the superiority of EvSSC under motion blur and extreme weather conditions, where autonomous driving is challenged. The established datasets and our codebase will be made publicly at https://github.com/Pandapan01/EvSSC.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。