用专家混合模型实现无标签3D点云多数据集联合训练
Point-MoE: Large-Scale Multi-Dataset Training with Mixture-of-Experts for 3D Semantic Segmentation
- 通过稀疏激活的专家MLP与轻量路由机制，让点云自主选择专用专家
- 在室内室外多数据集联合训练中超越现有方法，零样本泛化能力更强
- 适合做大规模3D语义分割且无需数据集标签的研究者
尽管在自然语言处理和2D视觉中大规模数据与模型的扩展已成主流，其在3D点云理解中的收益仍有限。本文研究在实际场景下扩展3D点云理解的初步步骤：在无数据集标签的情况下进行大规模多数据集联合训练以实现3D语义分割。点云来自多种传感器（如深度相机、LiDAR）和场景（如室内、室外），导致扫描模式、采样密度和语义偏差高度异质；直接混合这些数据会损害标准模型性能。为此，我们提出Point-MoE，一种基于专家混合的设计，通过稀疏激活的专家MLP和轻量级top-$k$路由机制扩展模型容量，使每个点可自主选择专用专家，无需数据集监督。在包含室内与室外数据集的混合数据上联合训练，并在已见数据集及零样本设置下评估，Point-MoE表现优于先前方法，且未使用任何数据集标签进行训练或推理。这为3D感知提供了一条可扩展路径：让模型自行发现异构3D数据中的结构，而非依赖人工标注或数据集特定启发式规则。
原文摘要 · Abstract (English)
While massively scaling both data and models have become central in NLP and 2D vision, their benefits for 3D point cloud understanding remain limited. We study the initial step of scaling 3D point cloud understanding under a realistic regime: large-scale multi-dataset joint training for 3D semantic segmentation, with no dataset labels available at training or inference time. Point clouds arise from a wide range of sensors (e.g., depth cameras, LiDAR) and scenes (\eg, indoor, outdoor), yielding heterogeneous scanning patterns, sampling densities, and semantic biases; naively mixing such datasets degrades standard models. Therefore, we introduce Point-MoE, a Mixture-of-Experts design that expands model capacity through sparsely activated expert MLPs and a lightweight top-$k$ router, allowing tokens to select specialized experts without requiring dataset supervision. Trained jointly on a diverse mix of indoor and outdoor datasets, and evaluated on seen datasets as well as in zero-shot settings, Point-MoE outperforms prior methods without using dataset labels for either training or inference. This outlines a scalable path for 3D perception: letting the model discover structure in heterogeneous 3D data rather than imposing it via manual curation or dataset-specific heuristics.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。