用伯努利噪声和得分函数生成二值数据,实现高效采样。
Sampling Binary Data by Denoising through Score Functions
- 以伯努利噪声替代高斯噪声,构建二值数据的去噪框架。
- 高噪声下采样等同于对数凹采样,计算效率显著提升。
- 多测量并行采样可降低有效噪声,适合图像二值化任务。
在欧氏空间中,高斯平滑与基于经验贝叶斯形式的去噪(即Tweedie-Miyasawa公式,TMF)是得分生成模型成功的关键。平滑降低高维学习难度,去噪恢复原始信号,而TMF通过噪声数据的得分函数将二者联系起来。本文将该范式拓展至布尔超立方体上的二值数据分布学习与采样,采用伯努利噪声作为平滑手段。我们推导出针对汉明损失的类TMF表达式,其中自然出现得分函数。通过类似朗之万的采样器实现噪声二值数据采样,并理论分析了不同噪声水平下的表现。在高伯努利噪声下,采样变得简单,类似于欧氏空间中的对数凹采样。此外,我们将Saremi等人(2024)的序列多测量采样方法扩展到二值场景,在固定噪声水平下通过采样多个噪声测量值降低有效噪声,无需连续时间随机过程。我们在合成数据和二值化图像上验证了该方法的理论结论与有效性。
原文摘要 · Abstract (English)
Gaussian smoothing combined with a probabilistic framework for denoising via the empirical Bayes formalism, i.e., the Tweedie-Miyasawa formula (TMF), are the two key ingredients in the success of score-based generative models in Euclidean spaces. Smoothing holds the key for easing the problem of learning and sampling in high dimensions, denoising is needed for recovering the original signal, and TMF ties these together via the score function of noisy data. In this work, we extend this paradigm to the problem of learning and sampling the distribution of binary data on the Boolean hypercube by adopting Bernoulli noise, instead of Gaussian noise, as a smoothing device. We first derive a TMF-like expression for the optimal denoiser for the Hamming loss, where a score function naturally appears. Sampling noisy binary data is then achieved using a Langevin-like sampler which we theoretically analyze for different noise levels. At high Bernoulli noise levels sampling becomes easy, akin to log-concave sampling in Euclidean spaces. In addition, we extend the sequential multi-measurement sampling of Saremi et al. (2024) to the binary setting where we can bring the "effective noise" down by sampling multiple noisy measurements at a fixed noise level, without the need for continuous-time stochastic processes. We validate our formalism and theoretical findings by experiments on synthetic data and binarized images.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。