提出解耦仲裁框架,解决多模态3D点云分割中的语义混淆问题。
Rethinking Multimodal Few-Shot 3D Point Cloud Segmentation: From Fused Refinement to Decoupled Arbitration
- 分离几何与语义路径,通过解耦对齐模块协同优化
- 在S3DIS和ScanNet上显著提升边界精度与纹理区分能力
- 适合需要高精度3D分割的工业检测与自动驾驶场景
本文重新审视多模态少样本3D点云语义分割(FS-PCS),发现传统‘融合后精炼’范式存在‘可塑性-稳定性困境’,且CLIP的类间混淆导致语义盲区。为此,我们提出解耦专家仲裁少样本分割网络(DA-FSS),有效区分语义与几何路径,并通过梯度相互正则化实现更好泛化。DA-FSS采用与MM-FSS相同的主干网络和预训练文本编码器生成文本嵌入,提高模态利用率并更好利用各模态信息空间。我们设计了并行专家精炼模块以生成各模态相关性,还提出堆叠仲裁模块(SAM)进行卷积融合与路径相关性仲裁。并行专家解耦两个路径:几何专家保持可塑性,语义专家确保稳定性。二者通过解耦对齐模块(DAM)实现知识迁移而不传播混淆。在S3DIS、ScanNet等主流数据集上的实验表明,DA-FSS优于MM-FSS,几何边界、完整性及纹理区分性能均更优。代码已开源:https://github.com/MoWenQAQ/DA-FSS/
原文摘要 · Abstract (English)
In this paper, we revisit multimodal few-shot 3D point cloud semantic segmentation (FS-PCS), identifying a conflict in "Fuse-then-Refine" paradigms: the "Plasticity-Stability Dilemma." In addition, CLIP's inter-class confusion can result in semantic blindness. To address these issues, we present the Decoupled-experts Arbitration Few-Shot SegNet (DA-FSS), a model that effectively distinguishes between semantic and geometric paths and mutually regularizes their gradients to achieve better generalization. DA-FSS employs the same backbone and pre-trained text encoder as MM-FSS to generate text embeddings, which can increase free modalities' utilization rate and better leverage each modality's information space. To achieve this, we propose a Parallel Expert Refinement module to generate each modal correlation. We also propose a Stacked Arbitration Module (SAM) to perform convolutional fusion and arbitrate correlations for each modality pathway. The Parallel Experts decouple two paths: a Geometric Expert maintains plasticity, and a Semantic Expert ensures stability. They are coordinated via a Decoupled Alignment Module (DAM) that transfers knowledge without propagating confusion. Experiments on popular datasets (S3DIS, ScanNet) demonstrate the superiority of DA-FSS over MM-FSS. Meanwhile, geometric boundaries, completeness, and texture differentiation are all superior to the baseline. The code is available at: https://github.com/MoWenQAQ/DA-FSS/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。