用户可交互地选中3D高斯点云中的物体,实现精准编辑。
ArtisanGS: Interactive Tools for Gaussian Splat Selection with AI and Human in the Loop
- 通过AI快速将2D选择掩码映射到3D高斯点云,支持实时修正。
- 在真实场景下实现任意二值分割,无需额外优化。
- 适合需要精细控制的3D内容编辑者,如动画与仿真应用。
3D高斯点云(3DGS)正成为传统图形学的可行替代方案,广泛应用于物理模拟和动画等领域。然而,从真实场景捕获中提取可用物体仍具挑战性,且对这一表示形式的可控编辑技术有限。不同于多数聚焦自动处理或高层编辑的现有方法,本文提出一套以高斯点云选择与分割为核心的交互式工具集。我们设计了一种快速的AI驱动方法,可将用户引导的2D选择掩码传播至3DGS,支持错误干预,并结合灵活的手动选择与分割工具,使用户能对非结构化3DGS场景实现几乎任意的二值分割。我们在现有高斯点云选择方法上进行评估,并通过构建自定义视频扩散模型,展示其在下游任务中的实用性。该工具可直接用于任意真实场景捕获,无需额外优化。
原文摘要 · Abstract (English)
Representation in the family of 3D Gaussian Splats (3DGS) are growing into a viable alternative to traditional graphics for an expanding number of application, including recent techniques that facilitate physics simulation and animation. However, extracting usable objects from in-the-wild captures remains challenging and controllable editing techniques for this representation are limited. Unlike the bulk of emerging techniques, focused on automatic solutions or high-level editing, we introduce an interactive suite of tools centered around versatile Gaussian Splat selection and segmentation. We propose a fast AI-driven method to propagate user-guided 2D selection masks to 3DGS selections. This technique allows for user intervention in the case of errors and is further coupled with flexible manual selection and segmentation tools. These allow a user to achieve virtually any binary segmentation of an unstructured 3DGS scene. We evaluate our toolset against the state-of-the-art for Gaussian Splat selection and demonstrate their utility for downstream applications by developing a user-guided local editing approach, leveraging a custom Video Diffusion Model. With flexible selection tools, users have direct control over the areas that the AI can modify. Our selection and editing tools can be used for any in-the-wild capture without additional optimization.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。