改用核密度估计损失，让矩阵感知更抗噪、优化更稳定。
Matrix Sensing with Kernel Optimal Loss: Robustness and Optimization Landscape
- 用核方法估计残差密度，最大化对数似然构造鲁棒损失。
- 在重尾噪声下仍能保持低误差点，理论证明可消除虚假极小值。
- 适合噪声分布未知或非高斯的机器学习任务，如医疗图像恢复。
本文研究非凸优化中损失函数的选择如何影响鲁棒性与优化景观，以含噪矩阵感知为例。传统回归常用均方误差（MSE）损失，但在非高斯或重尾噪声下表现不可靠。为此，我们采用基于非参数回归的鲁棒损失：利用核估计残差密度，并最大化估计对数似然。该损失在高斯误差下等价于MSE，但在更一般设定下仍保持稳定。进一步通过分析伪局部极小值消失的受限等距性质（RIP）常数上界，揭示该损失重塑优化景观的机制。理论与实证结果表明，新损失在大噪声和多样化噪声分布下均表现优异。本工作为通过简单更换损失函数提升机器学习鲁棒性提供了初步洞见，依托直观且普适的分析框架。
原文摘要 · Abstract (English)
In this paper we study how the choice of loss functions of non-convex optimization problems affects their robustness and optimization landscape, through the study of noisy matrix sensing. In traditional regression tasks, mean squared error (MSE) loss is a common choice, but it can be unreliable for non-Gaussian or heavy-tailed noise. To address this issue, we adopt a robust loss based on nonparametric regression, which uses a kernel-based estimate of the residual density and maximizes the estimated log-likelihood. This robust formulation coincides with the MSE loss under Gaussian errors but remains stable under more general settings. We further examine how this robust loss reshapes the optimization landscape by analyzing the upper-bound of restricted isometry property (RIP) constants for spurious local minima to disappear. Through theoretical and empirical analysis, we show that this new loss excels at handling large noise and remains robust across diverse noise distributions. This work offers initial insights into enhancing the robustness of machine learning tasks through simply changing the loss, guided by an intuitive and broadly applicable analytical framework.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。