融合PCA与DragGAN,实现高效可控的图像生成潜空间操作。
DragGANSpace: Latent Space Exploration and Control for GANs
- 用PCA压缩潜空间,结合DragGAN实现直观图像编辑。
- 在3层W+潜空间下优化时间减少,SSIM提升且视觉质量不变。
- 可对不同数据集训练的StyleGAN模型进行潜空间对齐与控制。
本工作将StyleGAN、DragGAN与主成分分析(PCA)相结合,提升GAN生成图像潜空间的效率与可控性。StyleGAN提供结构化潜空间,DragGAN支持直观图像操作,而PCA则降低维度并促进跨模型对齐,使潜空间探索更简洁可解释。我们以高质动物图像数据集(AFHQ)为例,发现将基于PCA的降维融入DragGAN框架,在保持性能的同时提升优化效率。尤其在浅层潜空间(W+层数=3)中,引入PCA可持续缩短总优化时间,同时维持良好视觉质量,甚至提升结构相似性指数(SSIM)。此外,我们展示了对两个训练于相似但不同数据域(AFHQ-Dog与AFHQ-Cat)的StyleGAN模型生成图像进行潜空间对齐的能力,并实现直观可解释的图像操控。结果表明该方法为图像合成与编辑提供了高效可解释的潜空间控制路径。
原文摘要 · Abstract (English)
This work integrates StyleGAN, DragGAN and Principal Component Analysis (PCA) to enhance the latent space efficiency and controllability of GAN-generated images. Style-GAN provides a structured latent space, DragGAN enables intuitive image manipulation, and PCA reduces dimensionality and facilitates cross-model alignment for more streamlined and interpretable exploration of latent spaces. We apply our techniques to the Animal Faces High Quality (AFHQ) dataset, and find that our approach of integrating PCA-based dimensionality reduction with the Drag-GAN framework for image manipulation retains performance while improving optimization efficiency. Notably, introducing PCA into the latent W+ layers of DragGAN can consistently reduce the total optimization time while maintaining good visual quality and even boosting the Structural Similarity Index Measure (SSIM) of the optimized image, particularly in shallower latent spaces (W+ layers = 3). We also demonstrate capability for aligning images generated by two StyleGAN models trained on similar but distinct data domains (AFHQ-Dog and AFHQ-Cat), and show that we can control the latent space of these aligned images to manipulate the images in an intuitive and interpretable manner. Our findings highlight the possibility for efficient and interpretable latent space control for a wide range of image synthesis and editing applications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。