用RGB图像实现布料3D状态精确估计，速度快精度高。
Cloth-Splatting: 3D Cloth State Estimation from RGB Supervision
- 结合动力学模型与3D高斯泼溅，实现状态预测与更新。
- 相比现有方法，状态估计更准且收敛更快。
- 仅需RGB图像监督，适合真实场景布料分析。
我们提出Cloth-Splatting，一种从RGB图像中估计布料3D状态的方法，采用预测-更新框架。该方法利用动作条件的动力学模型预测未来状态，并通过3D高斯泼溅更新预测结果。核心思想是将基于3D网格的表示与高斯泼溅结合，建立布料状态空间与图像空间之间的可微映射，从而在仅使用RGB监督的情况下，运用梯度优化技术修正不准确的状态估计。实验表明，Cloth-Splatting不仅显著提升状态估计精度，还有效缩短收敛时间。
原文摘要 · Abstract (English)
We introduce Cloth-Splatting, a method for estimating 3D states of cloth from RGB images through a prediction-update framework. Cloth-Splatting leverages an action-conditioned dynamics model for predicting future states and uses 3D Gaussian Splatting to update the predicted states. Our key insight is that coupling a 3D mesh-based representation with Gaussian Splatting allows us to define a differentiable map between the cloth state space and the image space. This enables the use of gradient-based optimization techniques to refine inaccurate state estimates using only RGB supervision. Our experiments demonstrate that Cloth-Splatting not only improves state estimation accuracy over current baselines but also reduces convergence time.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。