分离现实视频中物体与接触面,实现真实物理交互模拟
DecoupledGaussian: Object-Scene Decoupling for Physics-Based Interaction
- 通过联合泊松场修复分离后的物体与场景高斯分布
- 支持物体独立运动、碰撞与破碎,可响应用户施加的力
- 适合虚拟现实、机器人和自动驾驶中的真实感交互
我们提出 DecoupledGaussian,一个将现实视频中物体与其接触表面解耦的新系统,是实现基于牛顿物理的逼真模拟的关键前提。与以往依赖合成数据或仅在接触面弹性抖动的方法不同,DecoupledGaussian 允许物体在分离后大幅移动且不受初始接触面约束。针对现有2D图像修复工具难以恢复3D位置的问题,本方法提出联合泊松场来重建并扩展物体与接触场景的高斯表示。同时采用多切片策略优化物体几何形状。系统支持由用户指定冲量驱动的解耦运动、碰撞与断裂等复杂交互,适用于单个或跨多个场景的动态模拟。通过用户研究和定量基准测试验证了其有效性。该系统提升了真实环境中数字对象与场景的交互能力,广泛应用于虚拟现实、机器人及自动驾驶等领域。
原文摘要 · Abstract (English)
We present DecoupledGaussian, a novel system that decouples static objects from their contacted surfaces captured in-the-wild videos, a key prerequisite for realistic Newtonian-based physical simulations. Unlike prior methods focused on synthetic data or elastic jittering along the contact surface, which prevent objects from fully detaching or moving independently, DecoupledGaussian allows for significant positional changes without being constrained by the initial contacted surface. Recognizing the limitations of current 2D inpainting tools for restoring 3D locations, our approach proposes joint Poisson fields to repair and expand the Gaussians of both objects and contacted scenes after separation. This is complemented by a multi-carve strategy to refine the object's geometry. Our system enables realistic simulations of decoupling motions, collisions, and fractures driven by user-specified impulses, supporting complex interactions within and across multiple scenes. We validate DecoupledGaussian through a comprehensive user study and quantitative benchmarks. This system enhances digital interaction with objects and scenes in real-world environments, benefiting industries such as VR, robotics, and autonomous driving. Our project page is at: https://wangmiaowei.github.io/DecoupledGaussian.github.io/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。