神经网络通过反向展开实现贝叶斯推断，能逼近最优解且无需先验知识。
Unrolled denoising networks provably learn optimal Bayesian inference
- 将消息传递算法反向展开为神经网络，每层模拟一次迭代推断。
- 在压缩感知中，训练后网络层的去噪器逼近贝叶斯消息传递的最优去噪器。
- 适用于非高斯设计、低维和非乘积先验等复杂场景，适合研究推断算法者。
许多贝叶斯推断方法依赖于已知先验下的最优估计器设计。但当先验未知时，这些最优性保证有何意义？近年来，算法反向展开成为深度学习应对此问题的方案：设计一个神经网络，其层可模拟推断算法的迭代步骤，并在未知先验生成的数据上训练。尽管该方法表现良好，但其能否严格逼近已知先验下的最优性能仍不明确。本文首次为基于近似消息传递（AMP）反向展开的神经网络提供了严格的泛化保证。在压缩感知任务中，证明当训练数据来自乘积先验时，网络各层近似收敛至贝叶斯AMP所用的去噪器。此外，通过大量数值实验验证了该架构在压缩感知和秩一矩阵估计中的优势——不仅能无视先验适应通用分布，且在低维、非高斯设计和非乘积先验等更一般条件下优于贝叶斯AMP。
原文摘要 · Abstract (English)
Much of Bayesian inference centers around the design of estimators for inverse problems which are optimal assuming the data comes from a known prior. But what do these optimality guarantees mean if the prior is unknown? In recent years, algorithm unrolling has emerged as deep learning's answer to this age-old question: design a neural network whose layers can in principle simulate iterations of inference algorithms and train on data generated by the unknown prior. Despite its empirical success, however, it has remained unclear whether this method can provably recover the performance of its optimal, prior-aware counterparts. In this work, we prove the first rigorous learning guarantees for neural networks based on unrolling approximate message passing (AMP). For compressed sensing, we prove that when trained on data drawn from a product prior, the layers of the network approximately converge to the same denoisers used in Bayes AMP. We also provide extensive numerical experiments for compressed sensing and rank-one matrix estimation demonstrating the advantages of our unrolled architecture - in addition to being able to obliviously adapt to general priors, it exhibits improvements over Bayes AMP in more general settings of low dimensions, non-Gaussian designs, and non-product priors.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。