无需标注即可控制可动物体，用光流解耦运动与相机位移。
FreeGaussian: Annotation-free Control of Articulated Objects via 3D Gaussian Splats with Flow Derivatives
- 通过光流导数解耦相机运动与物体动作
- 实现无控制信号下的动态高斯点连续优化
- 3D球向量简化控制，适合复杂动作建模
从单目视频重建可控制的高斯点以表示可动物体极具挑战性，因约束不足。现有方法依赖密集掩码和人工定义的控制信号，限制了实际应用。本文提出免标注方法FreeGaussian，通过光流导数在数学上解耦相机自运动与物体动作。建立2D光流与3D高斯动态流之间的联系，使动态高斯运动能基于光流先验进行优化与连续性建模，无需任何控制信号。此外，引入3D球向量控制方案，将状态表示为3D高斯轨迹，避免复杂的一维控制信号计算，简化可控制高斯建模。在可动物体上的大量实验表明，该方法在视觉表现上达到当前最优，并具备精确且部件感知的可控性。代码已开源：https://github.com/Tavish9/freegaussian。
原文摘要 · Abstract (English)
Reconstructing controllable Gaussian splats for articulated objects from monocular video is especially challenging due to its inherently insufficient constraints. Existing methods address this by relying on dense masks and manually defined control signals, limiting their real-world applications. In this paper, we propose an annotation-free method, FreeGaussian, which mathematically disentangles camera egomotion and articulated movements via flow derivatives. By establishing a connection between 2D flows and 3D Gaussian dynamic flow, our method enables optimization and continuity of dynamic Gaussian motions from flow priors without any control signals. Furthermore, we introduce a 3D spherical vector controlling scheme, which represents the state as a 3D Gaussian trajectory, thereby eliminating the need for complex 1D control signal calculations and simplifying controllable Gaussian modeling. Extensive experiments on articulated objects demonstrate the state-of-the-art visual performance and precise, part-aware controllability of our method. Code is available at: https://github.com/Tavish9/freegaussian.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。