通过风格量化提升小样本生成对抗网络的图像一致性。
Style Quantization for Data-Efficient GAN Training
- 将连续潜空间映射到解耦风格空间,再用可学习码本进行离散化。
- 在有限数据下显著提升判别器鲁棒性和生成图像质量。
- 适合研究小样本生成、图像一致性优化的学者参考。
在数据稀缺场景下,生成对抗网络难以有效探索输入潜空间,导致邻近潜变量生成的图像在真实感上差异显著,影响一致性正则化(CR)效果。为此,我们提出SQ-GAN,通过引入风格空间量化机制增强CR。该方法将稀疏连续的输入潜空间转换为紧凑结构化的离散代理空间,使每个元素对应一个真实数据点,从而改善CR性能。不同于直接量化,我们先将输入潜变量映射至解耦的‘风格’空间,再利用可学习码本进行量化,使每个量化码控制独立的变异因素。此外,通过优化最优传输距离,将码本码与基础模型提取的训练数据特征对齐,融入外部知识,构建语义丰富的词汇表以准确描述训练数据集。大量实验表明,该方法显著提升了判别器鲁棒性和生成质量。
原文摘要 · Abstract (English)
Under limited data setting, GANs often struggle to navigate and effectively exploit the input latent space. Consequently, images generated from adjacent variables in a sparse input latent space may exhibit significant discrepancies in realism, leading to suboptimal consistency regularization (CR) outcomes. To address this, we propose \textit{SQ-GAN}, a novel approach that enhances CR by introducing a style space quantization scheme. This method transforms the sparse, continuous input latent space into a compact, structured discrete proxy space, allowing each element to correspond to a specific real data point, thereby improving CR performance. Instead of direct quantization, we first map the input latent variables into a less entangled ``style'' space and apply quantization using a learnable codebook. This enables each quantized code to control distinct factors of variation. Additionally, we optimize the optimal transport distance to align the codebook codes with features extracted from the training data by a foundation model, embedding external knowledge into the codebook and establishing a semantically rich vocabulary that properly describes the training dataset. Extensive experiments demonstrate significant improvements in both discriminator robustness and generation quality with our method.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。