用AI生成图像构建视觉蕴含数据集，效果接近真实数据。
Dataset Creation for Visual Entailment using Generative AI
- 以SNLI文本为提示，用Stable Diffusion生成对应图像构建新数据集。
- 在SNLI-VE和SICK-VTE上，合成数据训练模型F-score仅轻微下降。
- 适合数据稀缺场景下视觉蕴含模型的快速训练与验证。
本文提出并验证了一个用于训练视觉蕴含模型的新合成数据集。现有视觉蕴含数据集相较于文本蕴含数据集规模小且稀疏，人工构建成本高。本研究基于文本蕴含数据集SNLI，将其中的前提文本作为生成图像模型Stable Diffusion的输入提示，生成对应图像以替代原文字前提。我们通过内在与外在评估验证该数据集有效性：在外在评估中，使用基于CLIP特征向量的视觉蕴含分类器，以生成图像为训练数据，在SNLI-VE上获得F-score 0.686，略低于真实数据训练的0.703；在另一数据集SICK-VTE上，F-score从0.400降至0.384。结果表明，在数据稀缺情况下，合成数据可作为视觉蕴含模型训练的有效替代方案。
原文摘要 · Abstract (English)
In this paper we present and validate a new synthetic dataset for training visual entailment models. Existing datasets for visual entailment are small and sparse compared to datasets for textual entailment. Manually creating datasets is labor-intensive. We base our synthetic dataset on the SNLI dataset for textual entailment. We take the premise text from SNLI as input prompts in a generative image model, Stable Diffusion, creating an image to replace each textual premise. We evaluate our dataset both intrinsically and extrinsically. For extrinsic evaluation, we evaluate the validity of the generated images by using them as training data for a visual entailment classifier based on CLIP feature vectors. We find that synthetic training data only leads to a slight drop in quality on SNLI-VE, with an F-score 0.686 compared to 0.703 when trained on real data. We also compare the quality of our generated training data to original training data on another dataset: SICK-VTE. Again, there is only a slight drop in F-score: from 0.400 to 0.384. These results indicate that in settings with data sparsity, synthetic data can be a promising solution for training visual entailment models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。