提出新算法加速高秩矩阵感知,无需调参且收敛更快。
Efficient Over-parameterized Matrix Sensing from Noisy Measurements via Alternating Preconditioned Gradient Descent
- 交替更新因子矩阵,自动消除需调参的阻尼项
- 理论证明线性收敛至近最优误差,步长可更大
- 适用噪声矩阵感知等任务,实测效率领先
我们研究过参数化设定下的噪声矩阵感知问题,目标是从噪声观测中恢复一个真实秩为 $ r_ ext{star} $ 的矩阵 $ X_ ext{star} \in \mathbb{R}^{n_1 \times n_2} $,采用秩 $ r > r_ ext{star} $ 的因子分解 $ LR^\top $,其中 $ \min\{n_1, n_2\} \ge r $。现有预条件方法虽加速收敛,但依赖敏感的阻尼参数 $ \lambda $ 且对步长敏感。为此,我们提出交替预条件梯度下降(APGD)算法,交替更新两个因子矩阵,无需 $ \lambda $,支持更大步长,实现更快收敛。理论证明其以线性速率逼近最优误差,并可推广至加权PCA、1-bit矩阵补全、矩阵补全等低秩估计任务,均保持线性收敛。模拟与真实数据实验表明,APGD在各类低秩问题上均实现最快收敛速度与最低计算时间,性能显著优于现有方法。
原文摘要 · Abstract (English)
We consider the noisy matrix sensing problem in the over-parameterization setting, where the estimated rank $r$ is larger than the true rank $r_\star$ of the target matrix $X_\star$. Specifically, our main objective is to recover a matrix $ X_\star \in \mathbb{R}^{n_1 \times n_2} $ with rank $ r_\star $ from noisy measurements using an over-parameterized factorization $ LR^\top $, where $ L \in \mathbb{R}^{n_1 \times r}, \, R \in \mathbb{R}^{n_2 \times r} $ and $ \min\{n_1, n_2\} \ge r > r_\star $, with $ r_\star $ being unknown. Recently, preconditioning methods have been proposed to accelerate the convergence of matrix sensing problem compared to vanilla gradient descent, incorporating preconditioning terms $ (L^\top L + λI)^{-1} $ and $ (R^\top R + λI)^{-1} $ into the original gradient. However, these methods require careful tuning of the damping parameter $λ$ and are sensitive to step size. To address these limitations, we propose the alternating preconditioned gradient descent (APGD) algorithm, which alternately updates the two factor matrices, eliminating the need for the damping parameter $λ$ and enabling faster convergence with larger step sizes. We theoretically prove that APGD convergences to a near-optimal error at a linear rate. We further show that APGD can be extended to deal with other low-rank matrix estimation tasks, also with a theoretical guarantee of linear convergence. To validate the effectiveness and scalability of the proposed APGD, we conduct simulated and real-world experiments on a wide range of low-rank estimation problems, including noisy matrix sensing, weighted PCA, 1-bit matrix completion, and matrix completion. The extensive results demonstrate that APGD consistently achieves the fastest convergence and the lowest computation time compared to the existing alternatives.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。