用大规模稀疏配对提升流模型训练效率,显著改善图像生成效果。
On Fitting Flow Models with Large Sinkhorn Couplings
- 采用大批次规模的Sinkhorn算法生成高质量点对配对,优化速度场学习。
- 在合成数据与图像生成任务中,大批次+低正则化使生成质量显著提升。
- 适合追求高精度生成模型的研究者,尤其关注流模型训练优化者。
流模型通过时间依赖的速度场将数据从一种模态(如噪声)渐进转换到另一种(如图像)。当源点与目标点的配对已知时,训练可视为监督回归问题;但生成任务中无明确配对,训练难度大。现有方法独立采样源点与目标点,再通过最优传输(OT)求解器重排形成配对。近期工作多采用批量大小 $n /approx 256$,并使用匈牙利算法(尖锐耦合)或熵正则化Sinkhorn(模糊耦合)。本文探索将 $n$ 提升三至四数量级,并细致分析熵正则化参数 $\varepsilon$ 的影响。借助新的尺度不变性度量与跨多GPU分片计算,我们证明:在合成数据与图像生成任务中,使用大尺寸、低 $\varepsilon$ 的Sinkhorn耦合,能极大提升流模型性能,加速训练并降低推理开销。
原文摘要 · Abstract (English)
Flow models transform data gradually from one modality (e.g. noise) onto another (e.g. images). Such models are parameterized by a time-dependent velocity field, trained to fit segments connecting pairs of source and target points. When the pairing between source and target points is given, training flow models boils down to a supervised regression problem. When no such pairing exists, as is the case when generating data from noise, training flows is much harder. A popular approach lies in picking source and target points independently. This can, however, lead to velocity fields that are slow to train, but also costly to integrate at inference time. In theory, one would greatly benefit from training flow models by sampling pairs from an optimal transport (OT) measure coupling source and target, since this would lead to a highly efficient flow solving the Benamou and Brenier dynamical OT problem. In practice, recent works have proposed to sample mini-batches of $n$ source and $n$ target points and reorder them using an OT solver to form better pairs. These works have advocated using batches of size $n\approx 256$, and considered OT solvers that return couplings that are either sharp (using e.g. the Hungarian algorithm) or blurred (using e.g. entropic regularization, a.k.a. Sinkhorn). We follow in the footsteps of these works by exploring the benefits of increasing $n$ by three to four orders of magnitude, and look more carefully on the effect of the entropic regularization $\varepsilon$ used in the Sinkhorn algorithm. Our analysis is facilitated by new scale invariant quantities to report the sharpness of a coupling, while our sharded computations across multiple GPU or GPU nodes allow scaling up $n$. We show that in both synthetic and image generation tasks, flow models greatly benefit when fitted with large Sinkhorn couplings, with a low entropic regularization $\varepsilon$.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。