用2D掩码追踪解决3D实例分割过分割问题，提升精度和效率。
Any3DIS: Class-Agnostic 3D Instance Segmentation by 2D Mask Tracking
- 基于2D掩码追踪生成3D先验，避免无监督合并导致的冗余
- 动态规划筛选最优视角，优化超点形成精准3D提案
- 在ScanNet200/++上实现多类别、开放词汇与开放任务提升
现有3D实例分割方法常因过分割导致冗余且不准确的3D提案，根源在于其无监督的合并策略：将密集2D实例掩码跨帧映射到点云形成候选提案，再通过启发式规则分层合并，易产生大量冗余片段。为此，我们提出3D感知的2D掩码追踪模块，利用来自2D掩码分割与追踪基础模型（SAM-2）的鲁棒3D先验，确保视频帧间对象掩码的一致性。不同于将所有可见超点跨视图合并生成3D掩码，我们的3D掩码优化模块采用动态规划算法，选取最优视图组合，优化超点以生成每个对象的最终3D提案。该方法在场景中实现全面物体覆盖的同时，显著减少多余提案，有助于下游应用。在ScanNet200和ScanNet++上的评估表明，本方法在无类别、开放词汇与开放任务的3D实例分割任务中均取得显著提升。
原文摘要 · Abstract (English)
Existing 3D instance segmentation methods frequently encounter issues with over-segmentation, leading to redundant and inaccurate 3D proposals that complicate downstream tasks. This challenge arises from their unsupervised merging approach, where dense 2D instance masks are lifted across frames into point clouds to form 3D candidate proposals without direct supervision. These candidates are then hierarchically merged based on heuristic criteria, often resulting in numerous redundant segments that fail to combine into precise 3D proposals. To overcome these limitations, we propose a 3D-Aware 2D Mask Tracking module that uses robust 3D priors from a 2D mask segmentation and tracking foundation model (SAM-2) to ensure consistent object masks across video frames. Rather than merging all visible superpoints across views to create a 3D mask, our 3D Mask Optimization module leverages a dynamic programming algorithm to select an optimal set of views, refining the superpoints to produce a final 3D proposal for each object. Our approach achieves comprehensive object coverage within the scene while reducing unnecessary proposals, which could otherwise impair downstream applications. Evaluations on ScanNet200 and ScanNet++ confirm the effectiveness of our method, with improvements across Class-Agnostic, Open-Vocabulary, and Open-Ended 3D Instance Segmentation tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。