用3D高斯表示+骨骼驱动，实现视频中可编辑的动态物体建模。
RigGS: Rigging of 3D Gaussians for Modeling Articulated Objects in Videos
- 基于骨骼控制的高斯点变形，初始化动态物体模型。
- 无需模板先验，通过运动与语义简化生成稀疏骨骼结构。
- 支持新动作生成与高质量新视角渲染，适合动画与交互应用。
本文研究如何从2D视频中建模可动物体，以实现新视角合成，并具备易编辑、可驱动和可重用性。为解决这一挑战，我们提出RigGS，一种结合3D高斯表示与骨骼驱动运动表示的新范式，无需额外模板先验。首先，我们提出骨架感知的节点控制变形，随时间变形初始3D高斯表示，生成候选骨架节点，并根据其运动与语义信息简化为稀疏3D骨架。随后，基于该骨架设计可学习的皮肤变形与姿态依赖的细节变形，从而轻松调整3D高斯表示以生成新动作，并从新视角渲染高质量图像。大量实验表明，本方法能轻松生成真实感新动作并实现高质量渲染。
原文摘要 · Abstract (English)
This paper considers the problem of modeling articulated objects captured in 2D videos to enable novel view synthesis, while also being easily editable, drivable, and re-posable. To tackle this challenging problem, we propose RigGS, a new paradigm that leverages 3D Gaussian representation and skeleton-based motion representation to model dynamic objects without utilizing additional template priors. Specifically, we first propose skeleton-aware node-controlled deformation, which deforms a canonical 3D Gaussian representation over time to initialize the modeling process, producing candidate skeleton nodes that are further simplified into a sparse 3D skeleton according to their motion and semantic information. Subsequently, based on the resulting skeleton, we design learnable skin deformations and pose-dependent detailed deformations, thereby easily deforming the 3D Gaussian representation to generate new actions and render further high-quality images from novel views. Extensive experiments demonstrate that our method can generate realistic new actions easily for objects and achieve high-quality rendering.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。