arXiv:2505.19297cs.CV2025-05NeurIPS被引 5

用生成模型筛选高质量图文数据,打造小而精的训练集

Alchemist: Turning Public Text-to-Image Data into Generative Gold

  • 用预训练生成模型自动识别高价值训练样本
  • 3350条数据让5个开源模型生成质量显著提升
  • 适合想提升图像生成效果的研究者和开发者

预训练使文本到图像(T2I)模型具备广泛世界知识,但往往难以达到高审美质量与对齐效果。因此,监督微调(SFT)至关重要,其效果高度依赖微调数据集质量。现有公开SFT数据集多集中于特定领域(如动漫或特定艺术风格),构建高质量通用SFT数据集仍面临巨大挑战。当前筛选方法成本高,且难识别真正有影响力的样本。这一困境更因缺乏公开通用数据集而加剧,主流模型常依赖大规模、私有的内部数据,阻碍了研究进展。本文提出一种新方法,利用预训练生成模型作为高影响样本的估计器,构建并发布Alchemist——一个仅含3,350个样本但极为有效的通用SFT数据集。实验表明,Alchemist能显著提升五个公开T2I模型的生成质量,同时保持多样性和风格丰富性。此外,我们还公开了微调后模型的权重。

原文摘要 · Abstract (English)

Pre-training equips text-to-image (T2I) models with broad world knowledge, but this alone is often insufficient to achieve high aesthetic quality and alignment. Consequently, supervised fine-tuning (SFT) is crucial for further refinement. However, its effectiveness highly depends on the quality of the fine-tuning dataset. Existing public SFT datasets frequently target narrow domains (e.g., anime or specific art styles), and the creation of high-quality, general-purpose SFT datasets remains a significant challenge. Current curation methods are often costly and struggle to identify truly impactful samples. This challenge is further complicated by the scarcity of public general-purpose datasets, as leading models often rely on large, proprietary, and poorly documented internal data, hindering broader research progress. This paper introduces a novel methodology for creating general-purpose SFT datasets by leveraging a pre-trained generative model as an estimator of high-impact training samples. We apply this methodology to construct and release Alchemist, a compact (3,350 samples) yet highly effective SFT dataset. Experiments demonstrate that Alchemist substantially improves the generative quality of five public T2I models while preserving diversity and style. Additionally, we release the fine-tuned models' weights to the public.

文本生成图像数据集构建模型微调

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。