arXiv:2509.15060cs.LGcs.IT2025-09

提出新方法加速稀疏信号恢复,比传统方法快得多且无需采样。

Probabilistic and nonlinear compressive sensing

  • 用平滑概率模型替代l0正则,可直接计算梯度,无需蒙特卡洛采样。
  • 在多种信噪比下优于IHT和Lasso,在有限数据下仍表现稳健。
  • 揭示非线性压缩感知的深层限制,发现参数恢复存在对称性瓶颈。

我们提出一种平滑的概率重构方法,用于求解ℓ₀正则化回归问题,该方法无需蒙特卡洛采样即可精确计算梯度,显著加快了最佳子集选择问题的局部极值收敛速度。实验表明,该方法在广泛设定和信噪比条件下均优于IHT与(松弛)Lasso等压缩感知算法。实现可在CPU和GPU上高效运行,代码开源。此外,我们研究了非线性压缩感知中的参数恢复可行性,基于Fefferman和Markel定理,理论证明在无限数据极限下全局最优解可恢复至特定对称性范围内。通过设计正则化算法选取每类对称性代表,实验证明:尽管测试损失持续下降,但教师与学生网络配置在初期收敛后出现反向发散,呈现意外的回弹效应。这表明非线性压缩感知与线性情形存在根本差异。

原文摘要 · Abstract (English)

We present a smooth probabilistic reformulation of $\ell_0$ regularized regression that does not require Monte Carlo sampling and allows for the computation of exact gradients, facilitating rapid convergence to local optima of the best subset selection problem. The method drastically improves convergence speed compared to similar Monte Carlo based approaches. Furthermore, we empirically demonstrate that it outperforms compressive sensing algorithms such as IHT and (Relaxed-) Lasso across a wide range of settings and signal-to-noise ratios. The implementation runs efficiently on both CPUs and GPUs and is freely available at https://github.com/L0-and-behold/probabilistic-nonlinear-cs. We also contribute to research on nonlinear generalizations of compressive sensing by investigating when parameter recovery of a nonlinear teacher network is possible through compression of a student network. Building upon theorems of Fefferman and Markel, we show theoretically that the global optimum in the infinite-data limit enforces recovery up to certain symmetries. For empirical validation, we implement a normal-form algorithm that selects a canonical representative within each symmetry class. However, while compression can help to improve test loss, we find that exact parameter recovery is not even possible up to symmetries. In particular, we observe a surprising rebound effect where teacher and student configurations initially converge but subsequently diverge despite continuous decrease in test loss. These findings indicate fundamental differences between linear and nonlinear compressive sensing.

压缩感知非线性优化机器学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。