arXiv:2503.19486cs.CV2025-03

提出分阶段框架,实现人体图像视角、姿态、服装、身份的解耦控制。

Exploring Disentangled and Controllable Human Image Synthesis: From End-to-End to Stage-by-Stage

  • 分三步生成:先生成穿衣服的A姿势图,再合成背视图,最后控制姿态与视角。
  • 在野外数据上比端到端模型提升视觉质量与解耦效果,尤其适应真实场景。
  • 解决多数据源不一致问题,适合需要精细控制的人像生成应用。

在人体图像生成中实现细粒度可控性是计算机视觉的长期挑战。现有方法多聚焦于面部或近正面身体生成,难以同时解耦控制视角、姿态、服装和身份。本文提出一种新的解耦可控人体生成任务,在统一框架中显式分离并操控这四个因素。首先基于MVHumanNet训练端到端生成模型以实现因子解耦,但该模型在野外数据上表现不佳,因数据域差距导致性能下降。为更好利用数据,我们探索虚拟试衣(VTON)数据集作为补充,发现直接合并训练会因数据形式不一致破坏解耦过程。为此,提出分阶段框架,将生成分为三步:穿衣服的A姿势生成、背视图合成、姿态与视角控制。该结构化流程使不同阶段更有效利用数据,显著提升可控性与泛化能力,尤其适用于野外场景。大量实验表明,分阶段方法在视觉保真度与解耦质量上均优于端到端模型,为实际应用提供可扩展解决方案。更多演示见项目页:https://taited.github.io/discohuman-project/。

原文摘要 · Abstract (English)

Achieving fine-grained controllability in human image synthesis is a long-standing challenge in computer vision. Existing methods primarily focus on either facial synthesis or near-frontal body generation, with limited ability to simultaneously control key factors such as viewpoint, pose, clothing, and identity in a disentangled manner. In this paper, we introduce a new disentangled and controllable human synthesis task, which explicitly separates and manipulates these four factors within a unified framework. We first develop an end-to-end generative model trained on MVHumanNet for factor disentanglement. However, the domain gap between MVHumanNet and in-the-wild data produces unsatisfactory results, motivating the exploration of virtual try-on (VTON) dataset as a potential solution. Through experiments, we observe that simply incorporating the VTON dataset as additional data to train the end-to-end model degrades performance, primarily due to the inconsistency in data forms between the two datasets, which disrupts the disentanglement process. To better leverage both datasets, we propose a stage-by-stage framework that decomposes human image generation into three sequential steps: clothed A-pose generation, back-view synthesis, and pose and view control. This structured pipeline enables better dataset utilization at different stages, significantly improving controllability and generalization, especially for in-the-wild scenarios. Extensive experiments demonstrate that our stage-by-stage approach outperforms end-to-end models in both visual fidelity and disentanglement quality, offering a scalable solution for real-world tasks. Additional demos are available on the project page: https://taited.github.io/discohuman-project/.

人体生成解耦控制分阶段生成虚拟试衣

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。