不依赖骨骼绑定,用图像迁移生成高保真服装动画。
Learning High-Fidelity Cloth Animation via Skinning-Free Image Transfer
- 分离低频形状与高频褶皱,独立建模并直接监督。
- 通过2D图像迁移实现3D变形,恢复更精细的褶皱细节。
- 适用于多种服装拓扑,无需手动UV划分,适合虚拟试穿。
我们提出一种新方法,从给定人体姿态生成3D服装形变,对虚拟试穿和扩展现实等应用至关重要。现有方法多依赖线性混合皮肤(linear blend skinning)获取低频姿态服装形状,仅回归高频褶皱,但缺乏显式皮肤监督,常导致姿态错位,污染高频信号,难以恢复高保真褶皱。为此,我们提出无骨骼绑定方法,独立估计:(i) 低频姿态服装形状的顶点位置,(ii) 高频局部褶皱细节的顶点法向。通过解耦各频率模态,可直接以变形服装几何进行监督。为提升动画视觉质量,我们将顶点属性编码为渲染纹理图像,使3D服装形变等效于2D图像迁移,从而利用强大预训练图像模型恢复褶皱的细粒度视觉细节,同时保持对不同拓扑服装的优异可扩展性,无需人工UV划分。最后,提出多模态融合策略,结合双频率约束,从迁移图像中鲁棒重建变形3D服装。大量实验表明,该方法在多种服装类型上显著提升动画质量,恢复的褶皱比现有最优方法更精细。
原文摘要 · Abstract (English)
We present a novel method for generating 3D garment deformations from given body poses, which is key to a wide range of applications, including virtual try-on and extended reality. To simplify the cloth dynamics, existing methods mostly rely on linear blend skinning to obtain low-frequency posed garment shape and only regress high-frequency wrinkles. However, due to the lack of explicit skinning supervision, such skinning-based approach often produces misaligned shapes when posing the garment, consequently corrupts the high-frequency signals and fails to recover high-fidelity wrinkles. To tackle this issue, we propose a skinning-free approach by independently estimating posed (i) vertex position for low-frequency posed garment shape, and (ii) vertex normal for high-frequency local wrinkle details. In this way, each frequency modality can be effectively decoupled and directly supervised by the geometry of the deformed garment. To further improve the visual quality of animation, we propose to encode both vertex attributes as rendered texture images, so that 3D garment deformation can be equivalently achieved via 2D image transfer. This enables us to leverage powerful pretrained image models to recover fine-grained visual details in wrinkles, while maintaining superior scalability for garments of diverse topologies without relying on manual UV partition. Finally, we propose a multimodal fusion to incorporate constraints from both frequency modalities and robustly recover deformed 3D garments from transferred images. Extensive experiments show that our method significantly improves animation quality on various garment types and recovers finer wrinkles than state-of-the-art methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。