用高斯点云实现机器人数字孪生的实时仿真与视觉修正
GaussTwin: Unified Simulation and Correction with Gaussian Splatting for Robotic Digital Twins
- 结合物理动力学与高斯点云，实现真实世界与模拟环境统一建模
- 在Franka平台测试中追踪精度显著优于刚体与形变匹配基线
- 适合需要闭环控制与物理仿真的一体化机器人系统研究
数字孪生有望通过保持现实感知与仿真间的稳定关联来提升机器人操作性能。然而，现有系统普遍存在模型不统一、动态交互复杂及真实-仿真差距大等问题，限制了模型预测控制等下游应用。为此，我们提出GaussTwin，一种基于位置动力学与离散Cosserat杆模型的物理驱动仿真系统，结合高斯点云实现高效渲染与视觉修正。通过将高斯点锚定于物理实体，并依据光度误差与分割掩码驱动一致的SE(3)更新，GaussTwin实现了稳定的状态预测与修正，同时保持物理真实性。在仿真与Franka Research 3平台上实验表明，相比形状匹配与刚体基线，GaussTwin在追踪精度和鲁棒性上均有显著提升，并支持基于推动的规划等下游任务。结果表明，GaussTwin是迈向统一、物理可解释数字孪生的重要一步，可支撑闭环机器人交互与学习。
原文摘要 · Abstract (English)
Digital twins promise to enhance robotic manipulation by maintaining a consistent link between real-world perception and simulation. However, most existing systems struggle with the lack of a unified model, complex dynamic interactions, and the real-to-sim gap, which limits downstream applications such as model predictive control. Thus, we propose GaussTwin, a real-time digital twin that combines position-based dynamics with discrete Cosserat rod formulations for physically grounded simulation, and Gaussian splatting for efficient rendering and visual correction. By anchoring Gaussians to physical primitives and enforcing coherent SE(3) updates driven by photometric error and segmentation masks, GaussTwin achieves stable prediction-correction while preserving physical fidelity. Through experiments in both simulation and on a Franka Research 3 platform, we show that GaussTwin consistently improves tracking accuracy and robustness compared to shape-matching and rigid-only baselines, while also enabling downstream tasks such as push-based planning. These results highlight GaussTwin as a step toward unified, physically meaningful digital twins that can support closed-loop robotic interaction and learning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。