arXiv:2507.06146cs.CV2025-07IJCAI被引 1

无需提示词,用图像语义生成多物体图像,兼顾真实性和多样性。

Prompt-Free Conditional Diffusion for Multi-object Image Augmentation

  • 用局部-全局语义融合替代文本提示,从原图提取语义信息
  • 通过计数损失约束类别数量,生成图像更贴近真实数据分布
  • 提升下游任务性能与跨域泛化能力,适合数据增强场景

扩散模型已推动计算机视觉中数据增强的诸多进展。但在生成多物体图像以模拟真实场景时,现有方法要么完全依赖文本条件,导致生成物体与原始数据偏差;要么过度依赖原图,缺乏多样性,对下游任务帮助有限。为此,我们提出一种无提示词的条件扩散框架用于多物体图像增强。具体地,引入局部-全局语义融合策略,从图像中提取语义替代文本条件,并通过LoRA将知识注入扩散模型,缓解原模型与目标数据集间的类别偏差。同时设计基于计数的奖励模型,辅助传统重建损失进行训练。通过约束每类物体数量而非像素级约束,有效弥合生成数据与原始数据在数量上的差异,同时提升生成多样性。实验表明,该方法优于多个代表性先进基线,在下游任务中表现优异,并具备强跨域泛化能力。代码已公开。

原文摘要 · Abstract (English)

Diffusion models has underpinned much recent advances of dataset augmentation in various computer vision tasks. However, when involving generating multi-object images as real scenarios, most existing methods either rely entirely on text condition, resulting in a deviation between the generated objects and the original data, or rely too much on the original images, resulting in a lack of diversity in the generated images, which is of limited help to downstream tasks. To mitigate both problems with one stone, we propose a prompt-free conditional diffusion framework for multi-object image augmentation. Specifically, we introduce a local-global semantic fusion strategy to extract semantics from images to replace text, and inject knowledge into the diffusion model through LoRA to alleviate the category deviation between the original model and the target dataset. In addition, we design a reward model based counting loss to assist the traditional reconstruction loss for model training. By constraining the object counts of each category instead of pixel-by-pixel constraints, bridging the quantity deviation between the generated data and the original data while improving the diversity of the generated data. Experimental results demonstrate the superiority of the proposed method over several representative state-of-the-art baselines and showcase strong downstream task gain and out-of-domain generalization capabilities. Code is available at \href{https://github.com/00why00/PFCD}{here}.

图像增强扩散模型多物体生成无提示词

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。