无需噪声嵌入的去噪扩散模型,理论支持其在低维数据上表现更优。
Blind denoising diffusion models and the blessings of dimensionality
- 训练和采样时不输入噪声强度,简化模型设计
- 在低内在维度数据上性能接近标准扩散模型
- 适合追求简洁架构与理论保障的研究者
去噪扩散模型(DDMs)是跨领域学习数据分布的前沿方法,但其训练与采样流程仍缺乏深入理解。尤其噪声条件需人为添加噪声嵌入并设定随意的噪声调度。本文提出盲去噪扩散模型(BDDMs):训练与采样时不将噪声幅度输入神经网络,从而避免上述设计缺陷。我们证明,在底层数据分布的内在维度远低于环境维度的假设下,BDDMs 作为采样算法是正确的。该假设源于从单个噪声样本估计噪声水平的贝叶斯问题,可能具有独立研究价值。实验对比显示,基于分析严格支持的自适应方案显著优于标准 DDMs。
原文摘要 · Abstract (English)
Denoising diffusion models (DDMs) are state-of-the-art methods for learning densities from data across numerous domains, yet many aspects of the training and sampling pipeline remain poorly understood. In particular, noise conditioning requires practitioners to incorporate contrived unprincipled noise embeddings into neural network architectures and to use ad hoc noise schedules for sampling. To address these drawbacks, we provide a complete theory for \emph{blind denoising diffusion models} (BDDMs): a variant of DDMs where the noise amplitude is not passed into the neural network during training or sampling, obviating the need for the aforementioned design choices. We justify the correctness of BDDMs as a sampling algorithm under an assumption of low intrinsic dimensionality of the underlying data distribution relative to the ambient dimension. This assumption arises through the introduction of the Bayesian problem of estimating noise levels from a single noisy sample, which might be of independent interest. We empirically compare the performance of BDDMs to standard DDMs, showcasing the benefits of an \emph{adaptive} scheme which is rigorously justified by our analysis.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。