用少量真实图生成高保真合成数据,提升图像模型训练效果
LoFT: LoRA-fused Training Dataset Generation with Few-shot Guidance
- 通过微调LoRA并融合多张真实图特征生成合成数据
- 在10个数据集上验证,合成数据量达1000张/类时准确率显著提升
- 适合需要小样本生成高质量训练数据的研究者
尽管文本到图像生成技术取得进展,但合成数据对监督学习的性能提升有限,主要因合成数据难以忠实再现真实数据分布,缺乏保真度或多样性。现有方法虽引入少样本引导,仍无法捕捉特定真实图像的独特特征。本文提出新框架LoFT(LoRA-fused Training-data Generation with Few-shot Guidance),在单张真实图像上微调LoRA权重,并在推理时融合,生成结合多张真实图像特征的合成图像,提升数据多样性和保真度。我们在10个数据集上评估,每类使用8至64张真实图像作为引导,最多扩展至每类1000张合成图像。实验表明,基于LoFT生成的数据训练模型始终优于其他合成数据方法,且随着数据规模增大,准确率显著提升。分析显示,LoFT生成的数据具有高保真度和充足多样性,是性能提升的关键。代码已开源:https://github.com/ExplainableML/LoFT。
原文摘要 · Abstract (English)
Despite recent advances in text-to-image generation, using synthetically generated data seldom brings a significant boost in performance for supervised learning. Oftentimes, synthetic datasets do not faithfully recreate the data distribution of real data, i.e., they lack the fidelity or diversity needed for effective downstream model training. While previous work has employed few-shot guidance to address this issue, existing methods still fail to capture and generate features unique to specific real images. In this paper, we introduce a novel dataset generation framework named LoFT, LoRA-Fused Training-data Generation with Few-shot Guidance. Our method fine-tunes LoRA weights on individual real images and fuses them at inference time, producing synthetic images that combine the features of real images for improved diversity and fidelity of generated data. We evaluate the synthetic data produced by LoFT on 10 datasets, using 8 to 64 real images per class as guidance and scaling up to 1000 images per class. Our experiments show that training on LoFT-generated data consistently outperforms other synthetic dataset methods, significantly increasing accuracy as the dataset size increases. Additionally, our analysis demonstrates that LoFT generates datasets with high fidelity and sufficient diversity, which contribute to the performance improvement. The code is available at https://github.com/ExplainableML/LoFT.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。