用视觉与动态对齐，让仿真机器人直接落地真实场景。
TwinAligner: Visual-Dynamic Alignment Empowers Physics-aware Real2Sim2Real for Robotic Manipulation
- 通过SDF重建与可编辑3DGS实现像素级视觉对齐
- 从机器人-物体交互中识别刚体物理，保持动态一致
- 支持零样本泛化，适合需快速迭代的机器人研发
机器人领域正向数据驱动、端到端学习演进，受多模态大模型启发。但依赖昂贵的真实世界数据制约了进展。模拟器提供低成本替代方案，但仿真与现实间的差距阻碍策略迁移。本文提出TwinAligner，一种新型Real2Sim2Real系统，同时解决视觉与动态鸿沟。视觉对齐模块通过SDF重建和可编辑3DGS渲染实现像素级对齐；动态对齐模块通过识别机器人-物体交互中的刚体物理特性确保动态一致性。TwinAligner通过可扩展的数据收集与可信的迭代循环，加速算法开发。定量评估表明其在视觉与动态真实-仿真对齐方面表现优异，使仿真训练的策略在真实世界实现强零样本泛化。真实与仿真策略性能高度一致，凸显其推动可扩展机器人学习的潜力。代码与数据将公开于https://twin-aligner.github.io。
原文摘要 · Abstract (English)
The robotics field is evolving towards data-driven, end-to-end learning, inspired by multimodal large models. However, reliance on expensive real-world data limits progress. Simulators offer cost-effective alternatives, but the gap between simulation and reality challenges effective policy transfer. This paper introduces TwinAligner, a novel Real2Sim2Real system that addresses both visual and dynamic gaps. The visual alignment module achieves pixel-level alignment through SDF reconstruction and editable 3DGS rendering, while the dynamic alignment module ensures dynamic consistency by identifying rigid physics from robot-object interaction. TwinAligner improves robot learning by providing scalable data collection and establishing a trustworthy iterative cycle, accelerating algorithm development. Quantitative evaluations highlight TwinAligner's strong capabilities in visual and dynamic real-to-sim alignment. This system enables policies trained in simulation to achieve strong zero-shot generalization to the real world. The high consistency between real-world and simulated policy performance underscores TwinAligner's potential to advance scalable robot learning. Code and data will be released on https://twin-aligner.github.io
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。