arXiv:2505.11825cs.CVcs.AI2025-05

用不完整或损坏的数据训练扩散模型,提升数据利用效率。

Bootstrapping Diffusion: Diffusion Model Training Leveraging Partial and Corrupted Data

  • 将低分辨率、带水印等数据视为数据视图,分别训练模型后再融合
  • 理论证明:正则化后泛化误差更低,数据效率接近最优
  • 适合数据稀缺或质量不高的图像/视频生成场景

训练扩散模型需要大规模数据,但高质量数据(如高分辨率图像、长视频)获取困难。而大量低质量或部分数据(如低分辨率图像、带字幕/水印视频)常被丢弃。本文探讨能否利用这些部分数据训练传统扩散模型。基于理论分析,提出一种新方法:对每类部分数据分别训练一个扩散模型,再训练一个预测残差梯度函数的模型。理论上证明,采用合适正则化时,该方法可实现更低的泛化误差;残差训练难度与部分数据未捕捉的信号相关性成正比。结果表明,该方法在数据效率上接近一阶最优。

原文摘要 · Abstract (English)

Training diffusion models requires large datasets. However, acquiring large volumes of high-quality data can be challenging, for example, collecting large numbers of high-resolution images and long videos. On the other hand, there are many complementary data that are usually considered corrupted or partial, such as low-resolution images and short videos. Other examples of corrupted data include videos that contain subtitles, watermarks, and logos. In this study, we investigate the theoretical problem of whether the above partial data can be utilized to train conventional diffusion models. Motivated by our theoretical analysis in this study, we propose a straightforward approach of training diffusion models utilizing partial data views, where we consider each form of complementary data as a view of conventional data. Our proposed approach first trains one separate diffusion model for each individual view, and then trains a model for predicting the residual score function. We prove generalization error bounds, which show that the proposed diffusion model training approach can achieve lower generalization errors if proper regularizations are adopted in the residual score function training. In particular, we prove that the difficulty in training the residual score function scales proportionally with the signal correlations not captured by partial data views. Consequently, the proposed approach achieves near first-order optimal data efficiency.

扩散模型数据效率低质量数据

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。