通过自表示对齐提升3D点云表征,显著优化分割与检测性能。
Point-SRA: Self-Representation Alignment for 3D Representation Learning
- 采用多级掩码比与自蒸馏机制,捕捉几何与语义互补信息。
- 在ScanObjectNN上比Point-MAE提升5.37%,动脉分割达96.07% IoU。
- 适合点云分割、检测任务,尤其对复杂结构如动脉瘤有优势。
掩码自编码器(MAE)已成为3D表征学习的主流范式,在多个下游任务中创下新基准。现有方法使用固定掩码率,忽略多层次表征相关性与内在几何结构,且依赖点级重建假设,与点云多样性矛盾。为此,我们提出名为Point-SRA的3D表征学习方法,通过自蒸馏与概率建模对齐表征。具体地,为MAE分配不同掩码率以获取互补的几何与语义信息,同时利用均值流变换器(MFT)结合跨模态条件嵌入实现多样化概率重建。分析表明,MFT中不同时间步的表征也具互补性,因此在MAE与MFT层面分别设计双自表示对齐机制。最后,构建流条件微调架构,充分挖掘均值流所学点云分布。Point-SRA在ScanObjectNN上比Point-MAE提升5.37%;在颅内动脉瘤分割任务中,动脉平均交并比达96.07%,动脉瘤达86.87%;在3D物体检测任务中,达到47.3% AP@50,超越MaskPoint 5.12%。
原文摘要 · Abstract (English)
Masked autoencoders (MAE) have become a dominant paradigm in 3D representation learning, setting new performance benchmarks across various downstream tasks. Existing methods with fixed mask ratio neglect multi-level representational correlations and intrinsic geometric structures, while relying on point-wise reconstruction assumptions that conflict with the diversity of point cloud. To address these issues, we propose a 3D representation learning method, termed Point-SRA, which aligns representations through self-distillation and probabilistic modeling. Specifically, we assign different masking ratios to the MAE to capture complementary geometric and semantic information, while the MeanFlow Transformer (MFT) leverages cross-modal conditional embeddings to enable diverse probabilistic reconstruction. Our analysis further reveals that representations at different time steps in MFT also exhibit complementarity. Therefore, a Dual Self-Representation Alignment mechanism is proposed at both the MAE and MFT levels. Finally, we design a Flow-Conditioned Fine-Tuning Architecture to fully exploit the point cloud distribution learned via MeanFlow. Point-SRA outperforms Point-MAE by 5.37% on ScanObjectNN. On intracranial aneurysm segmentation, it reaches 96.07% mean IoU for arteries and 86.87% for aneurysms. For 3D object detection, Point-SRA achieves 47.3% AP@50, surpassing MaskPoint by 5.12%.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。