通过双流自回归建模,让虚拟人像的衣物动态更真实、更连贯。
DSAR: Dual-Stream Autoregressive Modeling of Temporal Cloth Dynamics for Photorealistic Animatable Avatars

- 分几何与状态双流,分别追踪表面变化和历史状态演化。
- 在复杂动作下仍保持衣物细节,时序一致性提升显著。
- 适合做高保真虚拟角色动画,尤其对非训练动作泛化强。
从RGB视频构建高保真且时序一致的可驱动人体虚拟形象仍具挑战。现有方法难以捕捉真实衣物动态,常导致外观过度平滑或在分布外姿态下出现严重伪影。根源在于忽视了衣物物理中的时序因果性:当前状态由前一帧状态经时间演化而来,而非仅依赖即时骨骼姿态。缺乏对这种因果结构的建模,网络学习的是姿态与外观的关联,而非运动演化过程,导致泛化能力差。本文提出一种双流自回归框架,显式建模可观测几何信息与隐含内部状态。几何流传播前一帧的表面位移,状态流融合当前特征与从记忆库中检索的历史状态。运动自适应聚合处理空间变化的动力学差异,自适应正则化平衡平滑性与灵活性。在多个挑战性数据集上的实验表明,该方法在渲染质量、时序一致性及对训练分布外动作模式的泛化能力上均有显著提升,验证了双流时序建模对实现真实衣物动态的有效性。
原文摘要 · Abstract (English)
Creating photorealistic and temporally coherent animatable human avatars from RGB videos remains challenging. Current methods struggle to capture realistic cloth dynamics, producing over-smoothed appearance or severe artifacts on out-of-distribution poses. This limitation stems from a fundamental oversight: existing approaches neglect the temporal causality inherent in cloth physics, where current states emerge from previous states through temporal evolution rather than instantaneous skeletal configurations alone. Without explicit modeling of this causal structure, networks learn pose-appearance correlations instead of motion evolution, leading to poor generalization. We introduce a dual-stream autoregressive framework that explicitly models both observable geometric information and implicit internal state. The geometric stream propagates surface displacement from the previous frame, while the state stream fuses current features with historical states retrieved from a memory bank. Motion-adaptive aggregation handles spatially-varying dynamics, and adaptive regularization balances smoothness with flexibility. Experiments on challenging datasets demonstrate significant improvements in rendering quality, temporal consistency, and generalization to motion patterns beyond training distributions, validating that dual-stream temporal modeling enables realistic cloth dynamics.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。