让3D高斯点在遮挡时保持物理合理性,靠的是可微分刚体模拟。
PersistGS: Differentiable Physics for Object Permanence in 4D Gaussian Splatting

- 用可微分刚体模拟推断遮挡期间物体运动轨迹
- 轨迹误差比光度监督低40%,在合成数据上接近真实轨迹
- 适合需要物理准确性的动态场景重建任务
动态3D高斯溅射(3DGS)方法通过多相机同步视频与光度监督重建时变场景。当运动物体完全被所有训练摄像头遮挡时,监督信号消失:代表该物体的高斯点无法获得梯度,导致退化。现有神经重建方法对不完整观测依赖学习到的生成先验,更注重视觉合理性而非物理正确性。我们提出PersistGS,通过将可微分刚体模拟与3DGS结合,恢复遮挡期间的物体恒存性。该方法将场景分解为对象级高斯点与碰撞网格,利用可微分模拟从遮挡前轨迹估计摩擦系数和速度,并基于得到的SE(3)轨迹在遮挡期内持续定位物体高斯点。因预测轨迹满足刚体动力学方程,能准确建模接触事件(弹跳、摩擦减速、方向变化),而运动学外推无法实现。我们引入质心轮廓损失,隔离位置梯度与外观噪声,使轨迹误差降低40%。在训练相机之外的视角观察遮挡过程的实验中,合成场景下,PersistGS相比恒定速度外推提升2.46dB PSNR,仅落后于真实轨迹上界0.19dB。
原文摘要 · Abstract (English)
Dynamic 3D Gaussian Splatting (3DGS) methods reconstruct time-varying scenes from synchronized multi-camera video using photometric supervision. When a moving object becomes fully occluded from all training cameras, this supervision vanishes: the Gaussians representing it receive no gradient signal and degrade. Existing approaches to incomplete observations in neural reconstruction rely on learned generative priors that prioritize visual plausibility over physical correctness. We propose $\textbf{PersistGS}$, a method that restores object permanence during occlusion by coupling differentiable rigid body simulation with 3D Gaussian Splatting. Our approach decomposes the scene into per-object Gaussians and collision meshes, estimates friction and velocity from the observed pre-occlusion trajectory via differentiable simulation, and uses the resulting SE(3) trajectory to position object Gaussians throughout the occlusion period. Because the predicted trajectory satisfies the governing equations of rigid body dynamics, it faithfully captures contact events (bounces, friction-based deceleration, direction changes) that kinematic extrapolation cannot model. We introduce a centroid silhouette loss that isolates positional gradients from appearance noise, yielding 40% lower trajectory error than photometric supervision. We evaluate using cameras withheld from training that observe the object during its occlusion. Experiments on synthetic scenes show that PersistGS outperforms constant velocity extrapolation by +2.46dB PSNR and comes within 0.19dB of a ground-truth trajectory upper bound.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。