用反事实图像生成自动构建图像检索训练三元组
Triplet Synthesis For Enhancing Composed Image Retrieval via Counterfactual Image Generation
- 通过反事实生成控制视觉特征变化，自动生成三元组
- 无需人工标注，可构建更大更丰富的训练数据集
- 提升图像检索模型性能，适合大规模视觉数据管理
组合图像检索（CIR）为管理和访问大规模视觉数据提供了一种有效方式。CIR模型的构建依赖于三元组，每个三元组包含一张参考图像、一段描述期望变化的修改文本，以及一张反映这些变化的目标图像。为有效训练CIR模型，需要大量人工标注来构建高质量训练数据集，这一过程耗时且费力。为此，本文提出一种新颖的三元组合成方法，利用反事实图像生成技术。通过控制反事实图像生成中的视觉特征变化，该方法可自动产生多样化的训练三元组，无需任何人工干预。该方法促进了更大、更具表现力的数据集的创建，从而提升了CIR模型的性能。
原文摘要 · Abstract (English)
Composed Image Retrieval (CIR) provides an effective way to manage and access large-scale visual data. Construction of the CIR model utilizes triplets that consist of a reference image, modification text describing desired changes, and a target image that reflects these changes. For effectively training CIR models, extensive manual annotation to construct high-quality training datasets, which can be time-consuming and labor-intensive, is required. To deal with this problem, this paper proposes a novel triplet synthesis method by leveraging counterfactual image generation. By controlling visual feature modifications via counterfactual image generation, our approach automatically generates diverse training triplets without any manual intervention. This approach facilitates the creation of larger and more expressive datasets, leading to the improvement of CIR model's performance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。