用少量样本实现稳定图像编辑,可精准控制属性而不改变类别
TAGE: Trustworthy Attribute Group Editing for Stable Few-shot Image Generation
- 通过离散代码本捕捉通用属性,支持跨类别编辑
- 仅需少量样本即可生成高质量、稳定的图像,优于现有方法
- 适合需要高效少样本图像生成的科研与设计场景
生成对抗网络(GAN)在图像编辑中表现突出,但现有方法依赖大量训练数据和显式标注,在新类别样本极少时难以有效操控多样属性。为此,我们提出TAGE,包含三个模块:代码本学习模块(CLM)、代码预测模块(CPM)和提示驱动语义模块(PSM)。CPM模块将类别无关属性编码至离散代码本中,基于图像由属性构成的思想,通过编辑这些属性可生成未见类别的图像。该模块通过预测代码本中属性向量的索引,实现自然图像编辑。同时,PSM模块生成语义提示,并融入CPM的Transformer结构,增强模型对目标属性的理解。借助语义提示,模型能在样本有限的情况下,更显著地突出期望属性,同时保持原始类别完整性。我们在Animal Faces、Flowers和VGGFaces数据集上进行了广泛实验,结果表明,所提方法不仅性能更优,且在少样本条件下具有更高稳定性,优于其他少样本图像生成技术。
原文摘要 · Abstract (English)
Generative Adversarial Networks (GANs) have emerged as a prominent research focus for image editing tasks, leveraging the powerful image generation capabilities of the GAN framework to produce remarkable results.However, prevailing approaches are contingent upon extensive training datasets and explicit supervision, presenting a significant challenge in manipulating the diverse attributes of new image classes with limited sample availability. To surmount this hurdle, we introduce TAGE, an innovative image generation network comprising three integral modules: the Codebook Learning Module (CLM), the Code Prediction Module (CPM) and the Prompt-driven Semantic Module (PSM). The CPM module delves into the semantic dimensions of category-agnostic attributes, encapsulating them within a discrete codebook. This module is predicated on the concept that images are assemblages of attributes, and thus, by editing these category-independent attributes, it is theoretically possible to generate images from unseen categories. Subsequently, the CPM module facilitates naturalistic image editing by predicting indices of category-independent attribute vectors within the codebook. Additionally, the PSM module generates semantic cues that are seamlessly integrated into the Transformer architecture of the CPM, enhancing the model's comprehension of the targeted attributes for editing. With these semantic cues, the model can generate images that accentuate desired attributes more prominently while maintaining the integrity of the original category, even with a limited number of samples. We have conducted extensive experiments utilizing the Animal Faces, Flowers, and VGGFaces datasets. The results of these experiments demonstrate that our proposed method not only achieves superior performance but also exhibits a high degree of stability when compared to other few-shot image generation techniques.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。