改进镜面下降法求解线性系统,突破无界域收敛难题。
Entropic Mirror Descent for Linear Systems: Polyak's Stepsize and Implicit Bias
- 采用类Polyak步长应对无界定义域的收敛挑战
- 实现ℓ₁范数隐式偏差更强上界及线性和次线性收敛
- 提出无需指数运算的新方法,理论保证收敛
本文研究将熵镜面下降法应用于求解线性系统,主要挑战来自定义域无界性。为在不施加严格假设条件下克服此问题,我们引入一种类Polyak型步长。在此过程中,我们强化了ℓ₁-范数隐式偏差的界,获得了次线性与线性收敛结果,并将收敛性推广至任意凸L-光滑函数。此外,我们提出一种替代方法,避免了指数运算,形式类似原始的Hadamard下降,但具有可证明的收敛性。
原文摘要 · Abstract (English)
This paper focuses on applying entropic mirror descent to solve linear systems, where the main challenge for the convergence analysis stems from the unboundedness of the domain. To overcome this without imposing restrictive assumptions, we introduce a variant of Polyak-type stepsizes. Along the way, we strengthen the bound for $\ell_1$-norm implicit bias, obtain sublinear and linear convergence results, and generalize the convergence result to arbitrary convex $L$-smooth functions. We also propose an alternative method that avoids exponentiation, resembling the original Hadamard descent, but with provable convergence.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。