arXiv:2502.10674cs.CV2025-02CVPR被引 4

针对3D物体识别中遮挡问题,提出新型预训练方法与轻量模型。

Occlusion-aware Text-Image-Point Cloud Pretraining for Open-World 3D Object Recognition

论文配图:Occlusion-aware Text-Image-Point Cloud Pretraining for Open-World 3D Object Recognition
图 1 · 摘自论文原文
  • 构建含遮挡的合成点云数据集,提升真实场景识别能力
  • 在14个类别上实现85.2%的零样本准确率,优于现有方法
  • 使用线性状态空间模型,降低计算开销,适合部署

当前开放世界3D表示学习多依赖CLIP实现零样本识别,但因预训练环境过于理想化,面对真实带遮挡点云时性能仍不理想。此外,基于Transformer注意力机制的方法推理成本过高。本文提出两个改进:首先,设计遮挡感知的文本-图像-点云联合预训练框架,从52,000个合成3D物体生成近63万部分点云用于训练,显著缩小了训练与测试之间的领域差距,持续提升主流3D网络在真实场景下的识别表现;其次,提出专为点云设计的双流线性状态空间模型DuoMamba,通过引入两条空间填充曲线结合一维卷积,有效建模点元间空间依赖关系,替代传统Transformer。在本框架下预训练的DuoMamba,在多个基准上超越现有最优方法,同时降低延迟与浮点运算量(FLOPs),展现出在实际应用中的巨大潜力。代码与数据已开源。

原文摘要 · Abstract (English)

Recent open-world representation learning approaches have leveraged CLIP to enable zero-shot 3D object recognition. However, performance on real point clouds with occlusions still falls short due to unrealistic pretraining settings. Additionally, these methods incur high inference costs because they rely on Transformer's attention modules. In this paper, we make two contributions to address these limitations. First, we propose occlusion-aware text-image-point cloud pretraining to reduce the training-testing domain gap. From 52K synthetic 3D objects, our framework generates nearly 630K partial point clouds for pretraining, consistently improving real-world recognition performances of existing popular 3D networks. Second, to reduce computational requirements, we introduce DuoMamba, a two-stream linear state space model tailored for point clouds. By integrating two space-filling curves with 1D convolutions, DuoMamba effectively models spatial dependencies between point tokens, offering a powerful alternative to Transformer. When pretrained with our framework, DuoMamba surpasses current state-of-the-art methods while reducing latency and FLOPs, highlighting the potential of our approach for real-world applications. Our code and data are available at https://ndkhanh360.github.io/project-occtip.

3D识别遮挡处理轻量化模型自监督学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。