用扩散模型在测试时修复损坏点云,不改原模型也能提升识别准确率。
Test-Time Adaptation of 3D Point Clouds via Denoising Diffusion Models
- 通过变分自编码器与去噪扩散过程,将噪声点云还原到源域分布。
- 在ShapeNet上达到新基准,对ModelNet40和ScanObjectNN也表现稳健。
- 无需微调模型参数,适合部署在传感器易受损的真实场景中。
3D点云的测试时自适应(TTA)对缓解真实场景中训练与测试数据间的差异至关重要,尤其在处理受损点云时。例如,激光雷达数据可能受传感器故障或环境因素影响,产生领域差距。在线适应模型以应对分布偏移十分关键,因为无法为所有可能变化进行训练。现有方法多依赖自监督学习或伪标签微调预训练模型,但可能导致源域知识遗忘,降低未来测试的泛化能力。本文提出一种新型3D测试时自适应方法——3DD-TTA(3D Denoising Diffusion Test-Time Adaptation)。该方法采用扩散策略,在不改变源模型参数的前提下,将输入点云样本适应至源域。首先使用变分自编码器(VAE)将受损点云编码为形状隐变量与隐点;随后对这些隐点施加高斯噪声,并通过去噪扩散过程更新形状隐变量与隐点,以保持生成样本的一致性与保真度,使其更接近源域分布。我们在ShapeNet上进行了广泛实验,并验证其在ModelNet40和ScanObjectNN上的泛化能力,结果达到当前最优水平。代码已开源: https://github.com/hamidreza-dastmalchi/3DD-TTA。
原文摘要 · Abstract (English)
Test-time adaptation (TTA) of 3D point clouds is crucial for mitigating discrepancies between training and testing samples in real-world scenarios, particularly when handling corrupted point clouds. LiDAR data, for instance, can be affected by sensor failures or environmental factors, causing domain gaps. Adapting models to these distribution shifts online is crucial, as training for every possible variation is impractical. Existing methods often focus on fine-tuning pre-trained models based on self-supervised learning or pseudo-labeling, which can lead to forgetting valuable source domain knowledge over time and reduce generalization on future tests. In this paper, we introduce a novel 3D test-time adaptation method, termed 3DD-TTA, which stands for 3D Denoising Diffusion Test-Time Adaptation. This method uses a diffusion strategy that adapts input point cloud samples to the source domain while keeping the source model parameters intact. The approach uses a Variational Autoencoder (VAE) to encode the corrupted point cloud into a shape latent and latent points. These latent points are corrupted with Gaussian noise and subjected to a denoising diffusion process. During this process, both the shape latent and latent points are updated to preserve fidelity, guiding the denoising toward generating consistent samples that align more closely with the source domain. We conduct extensive experiments on the ShapeNet dataset and investigate its generalizability on ModelNet40 and ScanObjectNN, achieving state-of-the-art results. The code has been released at \url{https://github.com/hamidreza-dastmalchi/3DD-TTA}.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。