提出稀疏代理注意力机制,提升点云模型的全局感知与细节提取能力。
SP$^2$T: Sparse Proxy Attention for Dual-stream Point Transformer
- 用顶点关联的稀疏采样确保几何多样的点云可靠建模。
- 表驱动相对偏置实现高效代理交互,计算开销低。
- 双路架构平衡局部与全局信息,适合大场景点云理解。
点云变换器通过扩大感受野在三维理解任务中取得显著进展,但进一步扩展会削弱分组注意力并降低细节特征提取能力。代理作为抽象表示可实现全局感受野,现有方法存在局限:全局代理导致大规模点云二次复杂度及位置模糊;局部代理则面临采样不可靠、代理交互效率低、局部-全局信息融合失衡等问题。为此,本文提出稀疏代理点变换器(SP²T),基于局部代理设计双流结构,包含三项创新:1)基于顶点关联的空间化代理采样,增强在几何多样点云上的鲁棒性;2)采用表驱动相对偏置的稀疏代理注意力,实现高效地图-归约式交互计算;3)双分支架构保持局部与全局信息的平衡融合。大量实验表明,SP²T在室内与室外三维理解基准上达到当前最优性能,延迟可控,相比SPoTr@S3DIS提升3.8% mIoU,相比[email protected]提升22.9% mIoU。
原文摘要 · Abstract (English)
Point transformers have demonstrated remarkable progress in 3D understanding through expanded receptive fields (RF), but further expanding the RF leads to dilution in group attention and decreases detailed feature extraction capability. Proxy, which serves as abstract representations for simplifying feature maps, enables global RF. However, existing proxy-based approaches face critical limitations: Global proxies incur quadratic complexity for large-scale point clouds and suffer positional ambiguity, while local proxy alternatives struggle with 1) Unreliable sampling from the geometrically diverse point cloud, 2) Inefficient proxy interaction computation, and 3) Imbalanced local-global information fusion; To address these challenges, we propose Sparse Proxy Point Transformer (SP$^{2}$T) -- a local proxy-based dual-stream point transformer with three key innovations: First, for reliable sampling, spatial-wise proxy sampling with vertex-based associations enables robust sampling on geometrically diverse point clouds. Second, for efficient proxy interaction, sparse proxy attention with a table-based relative bias effectively achieves the interaction with efficient map-reduce computation. Third, for local-global information fusion, our dual-stream architecture maintains local-global balance through parallel branches. Comprehensive experiments reveal that SP$^{2}$T sets state-of-the-art results with acceptable latency on indoor and outdoor 3D comprehension benchmarks, demonstrating marked improvement (+3.8% mIoU vs. SPoTr@S3DIS, +22.9% mIoU vs. [email protected]) compared to other proxy-based point cloud methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。