提出改进梯度下降法,实现低秩矩阵恢复的最优采样与快速收敛。
Scaled Gradient Descent for Ill-Conditioned Low-Rank Matrix Recovery with Optimal Sampling Complexity

- 引入缩放梯度下降,通过自适应调整步长提升优化效率。
- 理论证明达到最优采样复杂度O((n₁+n₂)r)和对数级迭代复杂度。
- 适用于病态矩阵,适合高维低秩数据恢复场景。
低秩矩阵恢复旨在从少于n₁n₂个线性测量中重建一个n₁×n₂的秩r矩阵。尽管已有多种算法具备良好理论保证,基于梯度下降的非凸方法因计算高效而广受欢迎。然而,这类方法通常存在两个局限:样本复杂度为O((n₁+n₂)r²),且在目标矩阵病态时,需O(κ log(1/ε))次迭代才能达到ε精度,收敛缓慢。近期研究发现,缩放梯度下降(ScaledGD)可将迭代复杂度降至O(log(1/ε)),但样本复杂度仍为子最优。相比之下,标准梯度下降在半正定(PSD)情形下通过虚拟序列技术可实现最优样本复杂度O((n₁+n₂)r),但迭代复杂度为O(κ² log(1/ε))。本文通过更精细分析证明,ScaledGD在一般低秩矩阵恢复问题中同时达到最优样本复杂度O((n₁+n₂)r)和改进的迭代复杂度O(log(1/ε))。数值实验进一步验证其在病态矩阵下的加速效果与最优采样能力。
原文摘要 · Abstract (English)
The low-rank matrix recovery problem seeks to reconstruct an unknown $n_1 \times n_2$ rank-$r$ matrix from $m$ linear measurements, where $m\ll n_1n_2$. This problem has been extensively studied over the past few decades, leading to a variety of algorithms with solid theoretical guarantees. Among these, gradient descent based non-convex methods have become particularly popular due to their computational efficiency. However, these methods typically suffer from two key limitations: a sub-optimal sample complexity of $O((n_1 + n_2)r^2)$ and an iteration complexity of $O(κ\log(1/ε))$ to achieve $ε$-accuracy, resulting in slow convergence when the target matrix is ill-conditioned. Here, $κ$ denotes the condition number of the unknown matrix. Recent studies show that a preconditioned variant of GD, known as scaled gradient descent (ScaledGD), can significantly reduce the iteration complexity to $O(\log(1/ε))$. Nonetheless, its sample complexity remains sub-optimal at $O((n_1 + n_2)r^2)$. In contrast, a delicate virtual sequence technique demonstrates that the standard GD in the positive semidefinite (PSD) setting achieves the optimal sample complexity $O((n_1 + n_2)r)$, but converges more slowly with an iteration complexity $O(κ^2 \log(1/ε))$. In this paper, through a more refined analysis, we show that ScaledGD achieves both the optimal sample complexity $O((n_1 + n_2)r)$ and the improved iteration complexity $O(\log(1/ε))$. Notably, our results extend beyond the PSD setting to general low-rank matrix recovery problem. Numerical experiments further validate that ScaledGD accelerates convergence for ill-conditioned matrices with the optimal sampling complexity.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。