arXiv:2606.21753cs.GRcs.AI2026-06中稿 · CVPR

让3D高斯点云资产参与复杂物理模拟,实现多类型物体联动交互。

Scene-Level Heterogeneous Physics Simulation with 3D Gaussian Splats

论文配图:Scene-Level Heterogeneous Physics Simulation with 3D Gaussian Splats
图 1 · 摘自论文原文
  • 将3DGS、网格、流体等统一抽象为物理粒子集,打通视觉与物理的桥梁。
  • 首次实现3DGS资产在大规模真实场景中与多种物理对象双向交互。
  • 适用于影视特效、虚拟制作等需高保真物理联动的场景级应用。

3D高斯点云(3DGS)虽已实现顶级逼真渲染,但其表示形式无法被物理引擎理解。现有物理增强方法仅限于孤立物体在理想平面上的单向交互,难以处理复杂静态几何或异构资产。本文提出首个框架,首次实现3DGS资产在场景级、异构、多求解器物理模拟中的参与。核心是表示抽象框架,将3DGS、虚拟网格、流体等统一转换为统一物理粒子集。该粒子集连同由场景捕获生成的静态碰撞边界,输入求解器无关的物理内核进行计算,结果再映射回各资产以驱动其视觉重建。此架构实现了可变形3DGS资产与流体、网格及大规模实拍静态环境之间的复杂双向交互,展现出此前无法实现的真实耦合现象。

原文摘要 · Abstract (English)

3D Gaussian Splatting (3DGS) has achieved state-of-the-art photorealistic rendering, but the representation gap prevents these assets from being physically interactive. Production-grade physics engines do not understand the 3DGS representation, while prior physics-for-3DGS methods are monolithic silos. These prior works are fundamentally limited, demonstrating only object-centric physics in isolated environments, such as on an ideal plane. They are incapable of interacting with complex static collision geometry or heterogeneous assets. We propose a novel framework that, for the first time, bridges this gap by enabling 3DGS assets to participate in scene-level, heterogeneous, multi-solver physical simulations. Our core contribution is a Representation Abstraction Framework that translates all diverse assets, including 3DGS, virtual meshes, and fluids, into a unified physical particle set. This abstraction is key to enabling complex behaviors, such as the non-rigid deformation of 3DGS assets, within a unified physics pipeline. This particle set, along with the static scene collision boundaries derived from scene capture, is processed within a solver-agnostic physics kernel. The physical results are then mapped back to drive each asset's specific visual reconstruction. This architecture unlocks capabilities impossible with prior art. We demonstrate complex, two-way interactions between deformable 3DGS assets, standard CG assets such as fluids and meshes, and large-scale captured static environments, showcasing realistic coupled phenomena that were previously unattainable.

3D高斯物理模拟异构交互视觉-物理融合

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。