仅用一张图就能让3D生成模型迁移到新领域,且保持高质量和多样性。
One-shot Generative Domain Adaptation in 3D GANs
- 选关键权重微调,结合四类损失函数实现快速适应。
- 在单张参考图下仍能生成高保真、多视角一致的3D图像。
- 支持零样本迁移与隐空间编辑,适合3D内容创作应用。
3D感知图像生成需要大量训练数据以保证训练稳定并降低过拟合风险。本文首次提出一项新任务——单样本3D生成域自适应(One-shot 3D Generative Domain Adaptation, GDA),旨在仅依赖单张参考图像,将预训练的3D生成器从一个领域迁移到新领域。该任务追求高保真度、大多样性、跨域一致性及多视角一致性。本文提出3D-Adapter,首个单样本3D GDA方法,实现多样且忠实的生成。方法通过谨慎选择有限权重集进行微调,并引入四种先进损失函数促进适应;同时采用高效的渐进式微调策略提升过程效果。三者协同使3D-Adapter在所有3D GDA目标属性上均表现优异,定量与定性结果均验证其有效性。此外,3D-Adapter可无缝扩展至零样本场景,并保留预训练生成器在隐空间中的插值、重构与编辑能力。代码将公开于 https://github.com/iceli1007/3D-Adapter。
原文摘要 · Abstract (English)
3D-aware image generation necessitates extensive training data to ensure stable training and mitigate the risk of overfitting. This paper first considers a novel task known as One-shot 3D Generative Domain Adaptation (GDA), aimed at transferring a pre-trained 3D generator from one domain to a new one, relying solely on a single reference image. One-shot 3D GDA is characterized by the pursuit of specific attributes, namely, high fidelity, large diversity, cross-domain consistency, and multi-view consistency. Within this paper, we introduce 3D-Adapter, the first one-shot 3D GDA method, for diverse and faithful generation. Our approach begins by judiciously selecting a restricted weight set for fine-tuning, and subsequently leverages four advanced loss functions to facilitate adaptation. An efficient progressive fine-tuning strategy is also implemented to enhance the adaptation process. The synergy of these three technological components empowers 3D-Adapter to achieve remarkable performance, substantiated both quantitatively and qualitatively, across all desired properties of 3D GDA. Furthermore, 3D-Adapter seamlessly extends its capabilities to zero-shot scenarios, and preserves the potential for crucial tasks such as interpolation, reconstruction, and editing within the latent space of the pre-trained generator. Code will be available at https://github.com/iceli1007/3D-Adapter.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。