arXiv:2505.11934cs.CV2025-05CVPR被引 11

用户仅需2D交互即可精准操控3D高斯场景,无需训练。

iSegMan: Interactive Segment-and-Manipulate 3D Gaussians

  • 基于视图间极线约束实现交互跨视角传播
  • 通过可见性投票机制从SAM获取2D分割结果
  • 支持多种编辑操作,适合快速3D场景编辑

3D高斯溅射(3DGS)的高效渲染与显式表示推动了3D场景编辑的发展。然而,现有方法在控制编辑区域和提供交互反馈方面存在不足,常导致意外结果。为弥补这一缺陷,我们提出交互式分割与操控3D高斯(iSegMan),仅需任意视角的简单2D用户交互即可完成。为实现跨视角交互传播,提出极线引导的交互传播(EIP),利用极线约束实现高效鲁棒匹配。为避免场景特定训练,进一步提出基于可见性的高斯投票(VGV),从SAM获取2D分割,并基于高斯可见性建立像素与3D高斯间的投票机制以提取区域。借助EIP与VGV的高效精准区域控制,构建操控工具箱,支持多种区域编辑功能,显著提升场景编辑的可控性、灵活性与实用性。大量实验验证了iSegMan在3D场景操控与分割任务中的优势。

原文摘要 · Abstract (English)

The efficient rendering and explicit nature of 3DGS promote the advancement of 3D scene manipulation. However, existing methods typically encounter challenges in controlling the manipulation region and are unable to furnish the user with interactive feedback, which inevitably leads to unexpected results. Intuitively, incorporating interactive 3D segmentation tools can compensate for this deficiency. Nevertheless, existing segmentation frameworks impose a pre-processing step of scene-specific parameter training, which limits the efficiency and flexibility of scene manipulation. To deliver a 3D region control module that is well-suited for scene manipulation with reliable efficiency, we propose interactive Segment-and-Manipulate 3D Gaussians (iSegMan), an interactive segmentation and manipulation framework that only requires simple 2D user interactions in any view. To propagate user interactions to other views, we propose Epipolar-guided Interaction Propagation (EIP), which innovatively exploits epipolar constraint for efficient and robust interaction matching. To avoid scene-specific training to maintain efficiency, we further propose the novel Visibility-based Gaussian Voting (VGV), which obtains 2D segmentations from SAM and models the region extraction as a voting game between 2D Pixels and 3D Gaussians based on Gaussian visibility. Taking advantage of the efficient and precise region control of EIP and VGV, we put forth a Manipulation Toolbox to implement various functions on selected regions, enhancing the controllability, flexibility and practicality of scene manipulation. Extensive results on 3D scene manipulation and segmentation tasks fully demonstrate the significant advantages of iSegMan. Project page is available at https://zhao-yian.github.io/iSegMan.

3D高斯交互编辑视觉分割实时操控

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。