arXiv:2510.02730cs.LGcs.CV2025-10被引 2

基于神经科学启发的乘法去噪模型,实现更符合生物规律的生成采样。

Dale meets Langevin: A Multiplicative Denoising Diffusion Model

论文配图:Dale meets Langevin: A Multiplicative Denoising Diffusion Model
图 1 · 摘自论文原文
  • 用几何布朗运动作前向噪声过程,构建乘法型得分匹配模型。
  • 提出两种采样器:无符号约束与保留符号的Dale-Langevin采样器。
  • 引入新型Hyvärinen得分,适用于非负数据生成,性能优于传统方法。

指数梯度下降(EGD)是一种受神经科学启发的优化算法,其收敛时突触权重服从对数正态分布,与实验观察一致。由于几何布朗运动在任意固定时间的边际分布也为对数正态,该性质揭示了EGD与基于GBM的随机过程间的天然联系。本文提出一种以GBM为前向加噪过程的乘法型得分生成模型,并推导出其在原始空间与对数变换空间中的反向时间SDE。通过离散化得到两种采样器:直接来自原始空间SDE的符号无关采样器,以及通过Lamperti变换获得的符号保持采样器,称作Dale-Langevin采样器。我们发现该框架与镜像Langevin动力学相关,驱动优化的凸函数恰好控制Dale-Langevin采样器。在加性噪声扩散模型中自然出现的标准Stein得分,在乘法设定下被替换为修正版:{ t Hyvärinen得分} $oldsymbol{x} igcirc abla /log p_{oldsymbol{X}}(oldsymbol{x})$。为此设计新的乘法去噪得分匹配目标(M-DSM),证明其等价于显式得分匹配损失,并包含非负得分匹配损失作为特例。在MNIST、Fashion-MNIST、Kuzushiji-MNIST和CIFAR-10上的实验验证了该框架的生成能力。

原文摘要 · Abstract (English)

Exponentiated gradient descent (EGD), a biologically motivated optimisation algorithm that respects Dale's law, produces log-normally distributed synaptic weights at convergence, in alignment with experimental observations in neuroscience. Since the marginal distribution of geometric Brownian motion (GBM) at any fixed time is log-normal, this convergence property reveals a natural connection between EGD and GBM-based stochastic processes. We propose a multiplicative score-based generative model with GBM as a forward noising process and derive its corresponding reverse-time SDE in both the ambient space and in the $\log$-transformed space. We derive two multiplicative samplers by discretising the corresponding reverse-time SDEs: a sign-agnostic sampler obtained directly from the ambient-space reverse-time SDE, and a sign-preserving sampler, which we refer to as the Dale-Langevin sampler, obtained via the Lamperti transform. We connect the framework to Mirrored Langevin Dynamics, showing that the convex function driving EGD in optimisation precisely governs the Dale-Langevin sampler. While the standard Stein score, defined as $\nabla \log p_{\boldsymbol{X}}(\boldsymbol{x})$ for a random vector $\boldsymbol{X}$ evaluated at $\boldsymbol{x}$, comes up naturally in the additive noise based diffusion models, in the multiplicative setting, we encounter a modified version of the Stein score for sampling, which we refer to as the {\it Hyvärinen score}: $\boldsymbol{x} \circ \nabla \log p_{\boldsymbol{X}}(\boldsymbol{x})$. To estimate the score, we propose a new multiplicative denoising score-matching objective (M-DSM), prove its equivalence to the multiplicative explicit score-matching loss and show that it subsumes the non-negative score matching loss. Experimental results on MNIST, Fashion-MNIST, Kuzushiji-MNIST, and CIFAR-10 to validate the generative capability of the proposed framework.

生成模型扩散模型神经科学得分匹配

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。