arXiv:2602.01844cs.CVcs.AI2026-02

仅用视觉数据学习布料动态,无需已知物理属性。

CloDS: Visual-Only Unsupervised Cloth Dynamics Learning in Unknown Conditions

  • 通过多视角视频重建布料3D几何,再训练动力学模型。
  • 在未知条件下实现强泛化,对遮挡和大形变鲁棒。
  • 适合无物理参数的布料模拟研究者使用。

深度学习在模拟复杂动态系统方面表现出色,但现有方法需已知物理属性作为监督或输入,限制了其在未知条件下的应用。为此,我们提出布料动力学定位(CDG)新场景,实现仅从多视角视觉观测中无监督学习布料动力学。进一步提出CloDS框架,采用三阶段流程:先进行视频到几何的定位,再基于定位后的网格训练动力学模型。为应对定位过程中的大非线性变形与严重自遮挡,引入双位置透明度调制机制,通过基于网格的高斯点阵实现2D观测与3D几何间的双向映射。该机制同时考虑高斯成分的绝对与相对位置。全面实验表明,CloDS能有效从视觉数据中学习布料动力学,并在未见配置下保持强泛化能力。代码与可视化结果见 https://github.com/whynot-zyl/CloDS。

原文摘要 · Abstract (English)

Deep learning has demonstrated remarkable capabilities in simulating complex dynamic systems. However, existing methods require known physical properties as supervision or inputs, limiting their applicability under unknown conditions. To explore this challenge, we introduce Cloth Dynamics Grounding (CDG), a novel scenario for unsupervised learning of cloth dynamics from multi-view visual observations. We further propose Cloth Dynamics Splatting (CloDS), an unsupervised dynamic learning framework designed for CDG. CloDS adopts a three-stage pipeline that first performs video-to-geometry grounding and then trains a dynamics model on the grounded meshes. To cope with large non-linear deformations and severe self-occlusions during grounding, we introduce a dual-position opacity modulation that supports bidirectional mapping between 2D observations and 3D geometry via mesh-based Gaussian splatting in video-to-geometry grounding stage. It jointly considers the absolute and relative position of Gaussian components. Comprehensive experimental evaluations demonstrate that CloDS effectively learns cloth dynamics from visual data while maintaining strong generalization capabilities for unseen configurations. Our code is available at https://github.com/whynot-zyl/CloDS. Visualization results are available at https://github.com/whynot-zyl/CloDS_video}.%\footnote{As in this example.

布料模拟无监督学习视觉重建

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。