用自筛选机制自动净化视觉语言数据，提升模型性能。
Data Selection Through Iterative Self-Filtering for Vision-Language Settings
- 通过迭代训练与数据筛选，让模型自主选择优质数据
- 在无需预训练模型或额外数据下，显著提升下游任务表现
- 适合大规模视觉语言数据清洗与自监督学习场景
大量高质量数据对神经网络训练至关重要，但在大规模场景下人工标注难以实现，导致数据集存在显著噪声。现有方法多依赖启发式规则、人工标注参考集或预训练模型来缓解此问题。本文提出一种新型自驱动方法：利用CLIP模型在不断演化、自我筛选的数据集上进行训练。该数据集包含高置信度的干净样本和分布多样性的代表性样本。通过迭代训练与数据筛选，逐步优化数据混合比例。实验表明，使用该方法过滤后的视觉语言数据可显著提升下游任务性能，且无需额外数据或预训练模型。
原文摘要 · Abstract (English)
The availability of large amounts of clean data is paramount to training neural networks. However, at large scales, manual oversight is impractical, resulting in sizeable datasets that can be very noisy. Attempts to mitigate this obstacle to producing performant vision-language models have so far involved heuristics, curated reference datasets, and using pre-trained models. Here we propose a novel, bootstrapped method in which a CLIP model is trained on an evolving, self-selected dataset. This evolving dataset constitutes a balance of filtered, highly probable clean samples as well as diverse samples from the entire distribution. Our proposed Self-Filtering method iterates between training the model and selecting a subsequently improved data mixture. Training on vision-language datasets filtered by the proposed approach improves downstream performance without the need for additional data or pre-trained models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。