用真实视频建软体数字孪生，实现高保真机器人操控评估
Real-to-Sim Robot Policy Evaluation with Gaussian Splatting Simulation of Soft-Body Interactions
- 基于真实视频重建软体物体，结合3D高斯泼溅实现视觉物理一体渲染
- 在毛绒玩具打包、绳索穿引等任务中，仿真结果与真实表现高度相关
- 适合需要高精度模拟软体交互的机器人策略验证与复现
机器人操作策略发展迅速，但直接在真实世界评估仍成本高、耗时长且难以复现，尤其涉及柔体物体的任务。仿真提供可扩展、系统化的替代方案，但现有模拟器常无法捕捉柔体交互的视觉与物理耦合复杂性。本文提出一种真实到仿真的策略评估框架，通过真实世界视频构建柔体数字孪生，并利用3D高斯泼溅技术以高保真度渲染机器人、物体和环境。我们在代表性柔体操作任务（包括毛绒玩具打包、绳索穿引、T型块推动）上验证该方法，结果显示仿真轨迹与真实执行性能高度相关，并揭示了学习策略的关键行为模式。结果表明，结合物理信息重建与高质量渲染，可实现可复现、可扩展、高准确的机器人操作策略评估。
原文摘要 · Abstract (English)
Robotic manipulation policies are advancing rapidly, but their direct evaluation in the real world remains costly, time-consuming, and difficult to reproduce, particularly for tasks involving deformable objects. Simulation provides a scalable and systematic alternative, yet existing simulators often fail to capture the coupled visual and physical complexity of soft-body interactions. We present a real-to-sim policy evaluation framework that constructs soft-body digital twins from real-world videos and renders robots, objects, and environments with photorealistic fidelity using 3D Gaussian Splatting. We validate our approach on representative deformable manipulation tasks, including plush toy packing, rope routing, and T-block pushing, demonstrating that simulated rollouts correlate strongly with real-world execution performance and reveal key behavioral patterns of learned policies. Our results suggest that combining physics-informed reconstruction with high-quality rendering enables reproducible, scalable, and accurate evaluation of robotic manipulation policies. Website: https://real2sim-eval.github.io/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。