从真实场景重建可模拟的物体形状与位置,支持复杂堆叠环境。
Simulation-Ready Cluttered Scene Estimation via Physics-aware Joint Shape and Pose Optimization
- 联合优化物体形状和姿态,引入物理接触约束
- 支持最多5个物体、22个凸包,重建结果可直接用于仿真
- 计算效率高,适合多物体交互场景的快速重建
从真实世界观测中估计可用于仿真的场景,对下游规划与策略学习至关重要。然而,现有方法在杂乱环境中表现不佳,普遍存在计算开销大、鲁棒性差、难以扩展至多个相互作用物体的问题。本文提出一种统一的基于优化的实时到仿真场景估计方法,联合恢复多个刚体物体在物理约束下的形状与位姿。核心创新包括:首先利用新提出的形状可微接触模型,实现几何与位姿的全局可微联合优化,并建模物体间接触关系;其次,通过挖掘增广拉格朗日海森矩阵的结构稀疏性,设计高效线性求解器,计算成本随场景复杂度呈良好增长。在此基础上,构建端到端的仿真就绪物理感知杂乱场景重建(SPARCS)流程,包含基于学习的物体初始化、物理约束下的联合形状-位姿优化及可微纹理精修。在最多包含5个物体和22个凸包的杂乱场景上实验表明,该方法能稳健重建物理合理、可直接用于仿真的物体形状与位姿。
原文摘要 · Abstract (English)
Estimating simulation-ready scenes from real-world observations is crucial for downstream planning and policy learning tasks. Regretfully, existing methods struggle in cluttered environments, often exhibiting prohibitive computational cost, poor robustness, and restricted generality when scaling to multiple interacting objects. We propose a unified optimization-based formulation for real-to-sim scene estimation that jointly recovers the shapes and poses of multiple rigid objects under physical constraints. Our method is built on two key technical innovations. First, we leverage the recently introduced shape-differentiable contact model, whose global differentiability permits joint optimization over object geometry and pose while modeling inter-object contacts. Second, we exploit the structured sparsity of the augmented Lagrangian Hessian to derive an efficient linear system solver whose computational cost scales favorably with scene complexity. Building on this formulation, we develop an end-to-end Simulation-ready Physics-Aware Reconstruction for Cluttered Scenes (SPARCS) pipeline, which integrates learning-based object initialization, physics-constrained joint shape-pose optimization, and differentiable texture refinement. Experiments on cluttered scenes with up to 5 objects and 22 convex hulls demonstrate that our approach robustly reconstructs physically valid, simulation-ready object shapes and poses. Project webpage: https://rory-weicheng.github.io/SPARCS/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。