用真实视频+少量3D数据训练,生成衣物360°新视角图像。
HoloGarment: 360° Novel View Synthesis of In-the-Wild Garments
- 融合真实视频与小规模合成3D数据,构建共享服装嵌入空间。
- 在真实视频上微调可生成无姿态依赖的完整衣物360°视图。
- 对褶皱、遮挡、姿势变化等现实问题鲁棒,细节逼真。
真实衣物的新视角合成(NVS)因严重遮挡、复杂人体姿态和布料形变而极具挑战。现有方法依赖大量未遮挡且静态的合成3D数据,难以泛化到真实场景。本文提出HoloGarment(全息衣物),仅需1-3张图像或连续视频,即可生成衣物在标准姿态下的360°新视角。核心思路是通过大规模真实视频与小规模合成3D数据结合,采用新型隐式训练范式,优化共享的服装嵌入空间。推理时,该嵌入空间支持从视频动态生成360°视图,通过微调特定视频的服装嵌入,构建独立于身体姿态与动作的“衣物地图”表示。实验表明,HoloGarment在真实衣物的图像与视频NVS任务上达到当前最优性能。方法能有效处理真实场景中的褶皱、姿态变化与遮挡,同时保持高保真度、视角一致性、精细纹理与准确几何结构。项目页展示更多结果:https://johannakarras.github.io/HoloGarment
原文摘要 · Abstract (English)
Novel view synthesis (NVS) of in-the-wild garments is a challenging task due significant occlusions, complex human poses, and cloth deformations. Prior methods rely on synthetic 3D training data consisting of mostly unoccluded and static objects, leading to poor generalization on real-world clothing. In this paper, we propose HoloGarment (Hologram-Garment), a method that takes 1-3 images or a continuous video of a person wearing a garment and generates 360° novel views of the garment in a canonical pose. Our key insight is to bridge the domain gap between real and synthetic data with a novel implicit training paradigm leveraging a combination of large-scale real video data and small-scale synthetic 3D data to optimize a shared garment embedding space. During inference, the shared embedding space further enables dynamic video-to-360° NVS through the construction of a garment "atlas" representation by finetuning a garment embedding on a specific real-world video. The atlas captures garment-specific geometry and texture across all viewpoints, independent of body pose or motion. Extensive experiments show that HoloGarment achieves state-of-the-art performance on NVS of in-the-wild garments from images and videos. Notably, our method robustly handles challenging real-world artifacts -- such as wrinkling, pose variation, and occlusion -- while maintaining photorealism, view consistency, fine texture details, and accurate geometry. Visit our project page for additional results: https://johannakarras.github.io/HoloGarment
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。