用AI生成无偏图像数据集,提升模型泛化能力。
Low-Biased General Annotated Dataset Generation
- 基于多模态模型,在语义空间中对齐类别描述生成图像
- 生成数据在多个任务上显著提升模型泛化性能
- 适合数据稀缺场景下的视觉模型预训练
在通用标注数据集(如ImageNet)上预训练主干网络已被证明对提升下游视觉任务的泛化能力至关重要。然而,人工标注图像常存在偏差,且无法跨类别或领域迁移,导致模型泛化能力下降。为此,本文提出低偏见通用标注数据集生成框架(lbGen),不依赖昂贵的人工收集,而是直接生成具有类别标注的低偏见图像。利用多模态基础模型(如CLIP)在由语言定义的低偏见语义空间中的对齐优势,设计双层语义对齐损失:一方面通过对抗学习使所有生成图像与目标数据集中所有类别的语义分布一致,另一方面要求每张图像与其类别名称的语义描述匹配。此外,将现有图像质量评分模型转为质量保证损失,以保持生成图像质量。仅需使用目标数据集的全部类别名称作为输入,即可通过微调预训练扩散模型获得低偏见图像生成模型。实验表明,相比人工标注数据集或其他合成数据集,使用本方法生成的数据能稳定提升多种主干网络在不同任务上的泛化能力,尤其在人工样本稀缺的任务中表现更优。
原文摘要 · Abstract (English)
Pre-training backbone networks on a general annotated dataset (e.g., ImageNet) that comprises numerous manually collected images with category annotations has proven to be indispensable for enhancing the generalization capacity of downstream visual tasks. However, those manually collected images often exhibit bias, which is non-transferable across either categories or domains, thus causing the model's generalization capacity degeneration. To mitigate this problem, we present a low-biased general annotated dataset generation framework (lbGen). Instead of expensive manual collection, we aim at directly generating low-biased images with category annotations. To achieve this goal, we propose to leverage the advantage of a multimodal foundation model (e.g., CLIP), in terms of aligning images in a low-biased semantic space defined by language. Specifically, we develop a bi-level semantic alignment loss, which not only forces all generated images to be consistent with the semantic distribution of all categories belonging to the target dataset in an adversarial learning manner, but also requires each generated image to match the semantic description of its category name. In addition, we further cast an existing image quality scoring model into a quality assurance loss to preserve the quality of the generated image. By leveraging these two loss functions, we can obtain a low-biased image generation model by simply fine-tuning a pre-trained diffusion model using only all category names in the target dataset as input. Experimental results confirm that, compared with the manually labeled dataset or other synthetic datasets, the utilization of our generated low-biased dataset leads to stable generalization capacity enhancement of different backbone networks across various tasks, especially in tasks where the manually labeled samples are scarce.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。