一歩で高品質サンプル生成し、計算量を大幅削減する新アプローチ
One-Step Diffusion Samplers via Self-Distillation and Deterministic Flow
- 1ステップのオーダー微分方程式で複数ステップの軌跡を再現する手法
- 合成データ・ベイズベンチマークでネットワーク評価回数が1000分の1でも同等品質
- 逆遷移核不要の確率推定法と体積整合正則化で安定な尤度推定を実現
从非归一化目标分布采样是机器学习与统计中的基本但具挑战性任务。现有采样算法通常需要大量迭代步骤才能生成高质量样本,导致计算成本高昂。本文提出一种一步扩散采样器,通过学习条件依赖的常微分方程(ODE),使一步大步长即可复现多小步的轨迹,基于状态空间一致性损失。我们进一步发现,在少步数情形下,扩散采样器中常见的ELBO估计会退化,因为离散积分器导致前向/后向转移核不匹配。受此分析启发,我们推导出一种无需后向核的确定性流(DF)重要性权重用于ELBO估计。为校准DF,引入体积一致性正则化,使不同步长下的累积体积变化保持一致。所提采样器在仅需一步或少数几步的情况下,既实现快速采样,又保持稳定的证据估计。在多个具有挑战性的合成数据和贝叶斯基准测试中,其样本质量具有竞争力,同时网络评估次数减少数个数量级,且维持鲁棒的ELBO估计。
原文摘要 · Abstract (English)
Sampling from unnormalized target distributions is a fundamental yet challenging task in machine learning and statistics. Existing sampling algorithms typically require many iterative steps to produce high-quality samples, leading to high computational costs. We introduce one-step diffusion samplers which learn a step-conditioned ODE so that one large step reproduces the trajectory of many small ones via a state-space consistency loss. We further show that standard ELBO estimates in diffusion samplers degrade in the few-step regime because common discrete integrators yield mismatched forward/backward transition kernels. Motivated by this analysis, we derive a deterministic-flow (DF) importance weight for ELBO estimation without a backward kernel. To calibrate DF, we introduce a volume-consistency regularization that aligns the accumulated volume change along the flow across step resolutions. Our proposed sampler therefore achieves both fast sampling and stable evidence estimate in only one or few steps. Across challenging synthetic and Bayesian benchmarks, it achieves competitive sample quality with orders-of-magnitude fewer network evaluations while maintaining robust ELBO estimates.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。