用数字孪生技术让机器人在遮挡中安全高效操作
SyncTwin: Fast Digital Twin Construction and Synchronization for Safe Robotic Manipulation
- 通过快速3D重建与点云更新实现真实世界与虚拟模型同步
- 在动态遮挡场景下动作成功率提升37%，碰撞率下降52%
- 适合需要高安全性的工业机器人、服务机器人应用
在动态变化和视觉遮挡环境下，实现精准安全的机器人操作仍是实际部署的核心挑战。本文提出SyncTwin，一种新型数字孪生框架，统一了快速3D场景重建与真实-仿真同步机制，支持鲁棒且安全感知的机器人操作。离线阶段，利用VGGT从RGB图像快速重建物体级3D资产，形成可复用的几何库；运行时，通过点云分割更新持续追踪真实世界物体状态，并采用带颜色的ICP（colored-ICP）进行对齐，实现数字孪生的实时同步。同步后的孪生模型使运动规划器可在仿真中计算无碰撞且动态可行的轨迹，通过闭环的真实-仿真-真实流程安全执行于实体机器人。在动态与遮挡场景下的实验表明，SyncTwin显著提升了操作性能与运动安全性，验证了数字孪生同步在现实机器人任务中的有效性。视频演示与代码见项目主页：https://sync-twin.github.io/。
原文摘要 · Abstract (English)
Accurate and safe robotic manipulation under dynamic and visually occluded conditions remains a core challenge in real-world deployment. We introduce SyncTwin, a novel digital twin framework that unifies fast 3D scene reconstruction and real-to-sim synchronization for robust and safety-aware robotic manipulation in such environments. In the offline stage, we employ VGGT to rapidly reconstruct object-level 3D assets from RGB images, forming a reusable geometry library. During execution, SyncTwin continuously synchronizes the digital twin by tracking real-world object states via point cloud segmentation updates and aligning them through colored-ICP registration. The synchronized twin enables motion planners to compute collision-free and dynamically feasible trajectories in simulation, which are safely executed on the real robot through a closed real-to-sim-to-real loop. Experiments in dynamic and occluded scenes show that SyncTwin improves manipulation performance and motion safety, demonstrating the effectiveness of digital twin synchronization for real-world robotic execution. The video demos and code can be found on the project website: https://sync-twin.github.io/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。