用物理约束重建杂乱场景，让机器人仿真更真实可靠。
Real-to-Sim for Highly Cluttered Environments via Physics-Consistent Inter-Object Reasoning
- 通过接触图建模物体间关系，联合优化位置与物理属性。
- 在真实和仿真环境中均实现高物理保真度，接触动态准确。
- 适合需要精确接触推理的机器人抓取与操作任务。
从单视角观测重建物理上有效的3D场景，是弥合视觉感知与机器人控制差距的关键。但在高度杂乱环境中的机器人操作等任务中，仅靠几何精度不足以满足精确接触推理需求。传统感知流程常忽略物理约束，导致物体悬浮或严重穿插，使下游仿真不可靠。为此，我们提出一种新的物理约束型Real-to-Sim管道，基于单视角RGB-D数据重建物理一致的3D场景。核心在于一个可微优化流程，通过接触图显式建模空间依赖，结合可微刚体仿真联合优化物体位姿与物理属性。在仿真与真实场景中的大量评估表明，所重建场景具有高物理保真度，能忠实复现真实接触动力学，支持稳定可靠的接触丰富型操作。
原文摘要 · Abstract (English)
Reconstructing physically valid 3D scenes from single-view observations is a prerequisite for bridging the gap between visual perception and robotic control. However, in scenarios requiring precise contact reasoning, such as robotic manipulation in highly cluttered environments, geometric fidelity alone is insufficient. Standard perception pipelines often neglect physical constraints, resulting in invalid states, e.g., floating objects or severe inter-penetration, rendering downstream simulation unreliable. To address these limitations, we propose a novel physics-constrained Real-to-Sim pipeline that reconstructs physically consistent 3D scenes from single-view RGB-D data. Central to our approach is a differentiable optimization pipeline that explicitly models spatial dependencies via a contact graph, jointly refining object poses and physical properties through differentiable rigid-body simulation. Extensive evaluations in both simulation and real-world settings demonstrate that our reconstructed scenes achieve high physical fidelity and faithfully replicate real-world contact dynamics, enabling stable and reliable contact-rich manipulation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。