通过自建噪声数据训练线性去噪器,实现比传统方法更优的去噪性能。
Precise Performance of Linear Denoisers in the Proportional Regime
- 用合成噪声数据直接训练线性去噪器,避免估计原始协方差
- 在样本与维度比趋近常数时,理论推导出去噪误差闭式解
- 数值实验显示优于经验维纳滤波,极限下逼近最优滤波器
本文研究了在形如 $\mathbf{x} + \mathbf{z}$ 的噪声数据上,线性去噪器的性能,其中 $\mathbf{x} \in \mathbb{R}^d$ 为均值为零、协方差未知的信号,$\mathbf{z} \sim \mathcal{N}(0, \mathbfΣ_{\mathbf{z}})$ 为加性高斯噪声。由于 $\mathbfΣ$ 未知,标准维纳滤波无法使用。我们假设拥有来自真实分布的样本 $\mathbf{x}_1,\dots,\mathbf{x}_n \in \mathbb{R}^d$。传统方法是估计 $\mathbfΣ$ 并构造经验维纳滤波器。但本文受扩散模型去噪步骤启发,另辟蹊径:通过向样本注入协方差为 $\mathbfΣ_1 \neq \mathbfΣ_{\mathbf{z}}$ 的高斯噪声,合成带噪样本 $\hat{\mathbf{x}}_i$,并训练线性去噪器 $\mathbf{W}$ 使 $\mathbf{W}\hat{\mathbf{x}}_i \approx \mathbf{x}_i$ 在最小二乘意义下最优。在比例渐近 $\frac{n}{d} \to \kappa > 1$ 下,利用凸高斯极小极大定理(CGMT)解析求得该去噪器泛化误差的闭式表达。基于此表达可优化 $\mathbfΣ_1$ 以获得最佳去噪器。数值实验表明,该方法在多种场景下优于经验维纳滤波,并随 $\kappa \to \infty$ 接近最优维纳滤波。
原文摘要 · Abstract (English)
In the present paper we study the performance of linear denoisers for noisy data of the form $\mathbf{x} + \mathbf{z}$, where $\mathbf{x} \in \mathbb{R}^d$ is the desired data with zero mean and unknown covariance $\mathbfΣ$, and $\mathbf{z} \sim \mathcal{N}(0, \mathbfΣ_{\mathbf{z}})$ is additive noise. Since the covariance $\mathbfΣ$ is not known, the standard Wiener filter cannot be employed for denoising. Instead we assume we are given samples $\mathbf{x}_1,\dots,\mathbf{x}_n \in \mathbb{R}^d$ from the true distribution. A standard approach would then be to estimate $\mathbfΣ$ from the samples and use it to construct an ``empirical" Wiener filter. However, in this paper, motivated by the denoising step in diffusion models, we take a different approach whereby we train a linear denoiser $\mathbf{W}$ from the data itself. In particular, we synthetically construct noisy samples $\hat{\mathbf{x}}_i$ of the data by injecting the samples with Gaussian noise with covariance $\mathbfΣ_1 \neq \mathbfΣ_{\mathbf{z}}$ and find the best $\mathbf{W}$ that approximates $\mathbf{W}\hat{\mathbf{x}}_i \approx \mathbf{x}_i$ in a least-squares sense. In the proportional regime $\frac{n}{d} \rightarrow κ> 1$ we use the {\it Convex Gaussian Min-Max Theorem (CGMT)} to analytically find the closed form expression for the generalization error of the denoiser obtained from this process. Using this expression one can optimize over $\mathbfΣ_1$ to find the best possible denoiser. Our numerical simulations show that our denoiser outperforms the ``empirical" Wiener filter in many scenarios and approaches the optimal Wiener filter as $κ\rightarrow\infty$.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。