提出旋转不变的点云自监督学习方法,提升模型对旋转变化的鲁棒性。
RI-MAE: Rotation-Invariant Masked AutoEncoders for Self-Supervised Point Cloud Representation Learning
- 设计旋转不变的RI-Transformer,解耦几何内容与相对方位信息。
- 采用双分支师生架构,在旋转不变隐空间中实现掩码点云重建。
- 在多个下游任务上达到当前最优,适合点云数据增强与自监督学习场景。
掩码点建模方法在点云自监督学习中取得显著进展,但对旋转敏感,常因旋转变化导致性能急剧下降。本文提出一种新型旋转不变掩码自编码器(RI-MAE),解决两大挑战:1)构建旋转不变的隐表示;2)在旋转不变空间中实现自监督重建。针对第一点,引入RI-Transformer,具备解耦几何内容、旋转不变的相对方位与位置嵌入机制,构建旋转不变的点云隐空间。针对第二点,设计新颖的双分支师生架构:教师编码所有点块,学生仅编码未掩码点块,通过额外的RI-Transformer预测器,利用学生输出预测被掩码点块的隐特征,并由教师输出监督。大量实验表明,该方法对旋转具有强鲁棒性,在多个下游任务上达到当前最优性能。代码已开源:https://github.com/kunmingsu07/RI-MAE。
原文摘要 · Abstract (English)
Masked point modeling methods have recently achieved great success in self-supervised learning for point cloud data. However, these methods are sensitive to rotations and often exhibit sharp performance drops when encountering rotational variations. In this paper, we propose a novel Rotation-Invariant Masked AutoEncoders (RI-MAE) to address two major challenges: 1) achieving rotation-invariant latent representations, and 2) facilitating self-supervised reconstruction in a rotation-invariant manner. For the first challenge, we introduce RI-Transformer, which features disentangled geometry content, rotation-invariant relative orientation and position embedding mechanisms for constructing rotation-invariant point cloud latent space. For the second challenge, a novel dual-branch student-teacher architecture is devised. It enables the self-supervised learning via the reconstruction of masked patches within the learned rotation-invariant latent space. Each branch is based on an RI-Transformer, and they are connected with an additional RI-Transformer predictor. The teacher encodes all point patches, while the student solely encodes unmasked ones. Finally, the predictor predicts the latent features of the masked patches using the output latent embeddings from the student, supervised by the outputs from the teacher. Extensive experiments demonstrate that our method is robust to rotations, achieving the state-of-the-art performance on various downstream tasks. Our code is available at https://github.com/kunmingsu07/RI-MAE.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。