用小波神经算子增强点云分割的全局上下文建模能力
PT-WNO: Point Transformer with Wavelet Neural Operator for 3D Point Cloud Semantic Segmentation

- 在编码器-解码器间引入可学习的小波神经算子,捕获多尺度全局频域特征
- 在S3DIS和DALES上分别提升1.03和1.47个点,显著改善语义分割精度
- 适合需要强全局理解的3D点云场景,尤其对细粒度分割任务有帮助
点云语义分割需同时捕捉精细局部几何与广泛全局场景结构。基于Transformer的网络虽在局部特征聚合上表现优异,但全局上下文主要依赖编码器-解码器间的跳跃连接,我们认为这不足以实现完整场景理解。我们假设,在跳跃连接基础上增加可学习的全局特征提取模块,使网络在深入局部细节前先获取场景级知识,从而生成更丰富、更具上下文感知的表示。为此,我们提出点变压器与小波神经算子(PT-WNO),在点云Transformer主干外并联一个共享的小波神经算子(WNO)分支。在每一层编码器-解码器转换时,点特征被投影到密集3D体网格中,由WNO通过可学习的小波分解与重构捕捉多尺度全局频域上下文。这些全局特征通过轻量适配器融合回网络,补充而非替代原有跳跃连接。在四个大规模3D点云基准测试上验证了该方法的有效性:在S3DIS(Area 5)上达到71.59% mIoU,较Point Transformer v3基线提升+1.03点;在DALES上达81.05% mIoU,提升+1.47点;在ScanNet~v2上获得76.19% mIoU,与基线(76.36%)保持竞争力。
原文摘要 · Abstract (English)
Point cloud semantic segmentation requires architectures that capture both fine-grained local geometry and broad global scene structure. Transformer-based networks have demonstrated strong performance by focusing on detailed local feature aggregation; however, global context is conveyed primarily through skip connections across encoder-decoder stages, which we argue is insufficient for full scene understanding. We hypothesize that augmenting skip connections with a learnable global feature extraction module allows the network to acquire scene-level knowledge before descending into local detail, leading to richer and more contextually grounded representations. To this end, we propose Point Transformer with Wavelet Neural Operato (PT-WNO), which integrates a shared Wavelet Neural Operator (WNO) branch alongside the skip connections of a point cloud transformer backbone. At each encoder-decoder transition, point features are projected onto a dense 3D volumetric grid where the WNO captures multi-scale global spectral context through learnable wavelet decomposition and reconstruction. These global features are fused back into the network via lightweight adapters, complementing rather than replacing the existing skip connections. Experiments on four large-scale 3D point cloud benchmarks demonstrate the effectiveness of PT-WNO. On S3DIS (Area 5), PT-WNO achieves 71.59% mIoU, outperforming the Point Transformer v3 (PTv3) baseline by +1.03 points. On DALES it achieves 81.05% mIoU (+1.47 over the baseline). On ScanNet~v2, PT-WNO obtains 76.19% mIoU, remaining competitive with the baseline (76.36%).
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。