用扩散模型降噪多智能体3D检测中的时空噪声,提升感知鲁棒性。
CoDiff: Conditional Diffusion Model for Collaborative 3D Object Detection
- 将特征投影到预训练编码器的隐空间,以扩散模型逐步去噪融合特征
- 在模拟与真实数据集上均超越现有方法,高噪声下仍保持性能优势
- 首个将扩散模型用于多智能体协同感知的工作,适合自动驾驶研究者
协同3D目标检测在自动驾驶中至关重要,能通过多智能体间信息共享显著提升个体感知能力。然而实际中,由于位姿估计误差和时延,跨智能体信息融合常引入时空噪声,导致检测错误。扩散模型具备从噪声样本恢复理想数据的能力,这启发我们探索其在多智能体协同感知中的应用。本文提出CoDiff,一种新颖的鲁棒协同感知框架,利用扩散模型生成更完整清晰的特征表示。据我们所知,这是首个将扩散模型应用于多智能体协同感知的工作。具体地,将高维特征图投影至强大预训练自编码器的隐空间,以各智能体信息为条件引导扩散采样过程,逐步去噪并优化融合特征。在模拟与真实数据集上的实验表明,所提框架CoDiff在协同目标检测性能上持续优于现有方法,并在智能体位姿与延迟信息含高噪声条件下表现出极强鲁棒性。代码已开源:https://github.com/HuangZhe885/CoDiff
原文摘要 · Abstract (English)
Collaborative 3D object detection holds significant importance in the field of autonomous driving, as it greatly enhances the perception capabilities of each individual agent by facilitating information exchange among multiple agents. However, in practice, due to pose estimation errors and time delays, the fusion of information across agents often results in feature representations with spatial and temporal noise, leading to detection errors. Diffusion models naturally have the ability to denoise noisy samples to the ideal data, which motivates us to explore the use of diffusion models to address the noise problem between multi-agent systems. In this work, we propose CoDiff, a novel robust collaborative perception framework that leverages the potential of diffusion models to generate more comprehensive and clearer feature representations. To the best of our knowledge, this is the first work to apply diffusion models to multi-agent collaborative perception. Specifically, we project high-dimensional feature map into the latent space of a powerful pre-trained autoencoder. Within this space, individual agent information serves as a condition to guide the diffusion model's sampling. This process denoises coarse feature maps and progressively refines the fused features. Experimental study on both simulated and real-world datasets demonstrates that the proposed framework CoDiff consistently outperforms existing relevant methods in terms of the collaborative object detection performance, and exhibits highly desired robustness when the pose and delay information of agents is with high-level noise. The code is released at https://github.com/HuangZhe885/CoDiff
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。