arXiv:2410.01801cs.CVcs.AI2024-10中稿 · SIGGRAPH被引 2

从任意衣物图生成高保真3D服装,保留纹理细节与材质特性。

FabricDiffusion: High-Fidelity Texture Transfer for 3D Garments Generation from In-The-Wild Clothing Images

  • 基于缝制图案思路,将纹理转为可重复平铺的无畸变贴图。
  • 在合成数据上训练扩散模型,修复输入图像中的扭曲和遮挡。
  • 兼容物理渲染流程,支持真实光照下的材质表现,适合工业级设计。

我们提出FabricDiffusion,一种将单张自然衣物图像中的织物纹理迁移至任意形状3D服装的方法。现有方法多通过2D到3D贴图映射或生成式深度感知补全合成纹理,但常因输入图像中遮挡、形变或姿态问题难以保留细节。受时尚产业中多数服装由平面可重复纹理拼接而成的启发,我们将纹理迁移任务转化为提取无畸变、可平铺的纹理材料,并映射至服装的UV空间。基于此,我们在大规模合成数据上训练去噪扩散模型,用于修正输入纹理图像中的形变。该过程生成平坦纹理贴图,可与现有基于物理的渲染(PBR)材质生成流程紧密集成,实现不同光照条件下的真实感重光照。实验表明,FabricDiffusion能有效传递纹理图案、材质属性及精细印花与标志,显著优于当前最优方法,在合成数据与真实世界衣物图像上均表现优异,且对未见纹理与服装形状具有良好泛化能力。

原文摘要 · Abstract (English)

We introduce FabricDiffusion, a method for transferring fabric textures from a single clothing image to 3D garments of arbitrary shapes. Existing approaches typically synthesize textures on the garment surface through 2D-to-3D texture mapping or depth-aware inpainting via generative models. Unfortunately, these methods often struggle to capture and preserve texture details, particularly due to challenging occlusions, distortions, or poses in the input image. Inspired by the observation that in the fashion industry, most garments are constructed by stitching sewing patterns with flat, repeatable textures, we cast the task of clothing texture transfer as extracting distortion-free, tileable texture materials that are subsequently mapped onto the UV space of the garment. Building upon this insight, we train a denoising diffusion model with a large-scale synthetic dataset to rectify distortions in the input texture image. This process yields a flat texture map that enables a tight coupling with existing Physically-Based Rendering (PBR) material generation pipelines, allowing for realistic relighting of the garment under various lighting conditions. We show that FabricDiffusion can transfer various features from a single clothing image including texture patterns, material properties, and detailed prints and logos. Extensive experiments demonstrate that our model significantly outperforms state-to-the-art methods on both synthetic data and real-world, in-the-wild clothing images while generalizing to unseen textures and garment shapes.

3D服装生成纹理迁移扩散模型PBR渲染

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。