量子算法在重尾噪声优化中实现速度提升,适用于低维场景。
Quantum Speedups for Stochastic Optimization with Heavy-Tailed Noise
- 设计新型量子均值估计算法,降低查询复杂度。
- 量子方法在低维下达到近最优,优于经典下界。
- 适合研究量子优化与高维统计学习的学者参考。
我们研究带有重尾梯度噪声的随机优化问题。提出一种针对多维重尾随机变量的新颖量子均值估计算法,在低维情形下查询复杂度低于最优经典估计器。通过广义多级蒙特卡洛技术,构建无偏量子均值估计。证明量子下界表明:当随机向量维度 $d$ 较小且视为常数时,我们的量子估计器在对数因子内最优。对于尾指数 $p>4/3$,推导出更强的维度依赖下界,说明低维情形下维度依赖不可避免。基于这些估计器,提出量子归一化随机梯度下降(QNSGD),以 $ ilde{ m O}ig( oot extstyle d elaxulletε^{-rac{5p-4}{2p-2}}ig)$ 次量子随机梯度预言机查询找到 $ε$-平稳点。对凸目标函数,提出量子投影随机梯度下降(QPSGD),期望查询次数为 $ ilde{ m O}ig( oot extstyle d elaxulletε^{-rac{3p-2}{2p-2}}+ε^{-2}ig)$。这些更紧上界优于经典下界 $Ωig(ε^{-rac{3p-2}{p-1}}ig)$(非凸)和 $Ωig(ε^{-rac{p}{p-1}}ig)$(凸),在低维情形 $d riangleq ε^{-rac{p}{p-1}}$ 与 $d riangleq ε^{-rac{2-p}{p-1}}$ 下成立。
原文摘要 · Abstract (English)
We study stochastic optimization with heavy-tailed gradient noise. We first propose a novel quantum mean estimator for multivariate heavy-tailed random variables that achieves lower query complexity than optimal classical estimators in the low-dimensional regime. We further develop an unbiased quantum mean estimator by applying a generalized multi-level Monte Carlo technique. We prove quantum lower bounds showing that, when the dimension $d$ of the random vector is small and can be viewed as a constant, our quantum estimators are optimal up to logarithmic factors. We further derive stronger dimension-dependent lower bounds for tail index $p>4/3$, showing that a nontrivial dependence on the dimension is unavoidable in the low-dimensional regime. Based on these estimators, we propose a quantum normalized stochastic gradient descent method ($\texttt{QNSGD}$), which finds an $ε$-stationary point using $\tilde{\mathcal{O}}\big(\sqrt d\,ε^{-\frac{5p-4}{2p-2}}\big)$ queries to the quantum stochastic gradient oracle. For a convex objective function, we propose a quantum projected stochastic gradient descent method ($\texttt{QPSGD}$), which computes a solution with $ε$-optimal solution using $\tilde{\mathcal{O}}\big(\sqrt d\,ε^{-\frac{3p-2}{2p-2}}+ε^{-2}\big)$ queries in expectation. These sharper bounds improve upon the classical lower bounds $Ω\big(ε^{-\frac{3p-2}{p-1}}\big)$ for nonconvex problems and $Ω\big(ε^{-\frac{p}{p-1}}\big)$ for convex problems in the low-dimensional regimes $d\lesssimε^{-\frac{p}{p-1}}$ and $d\lesssimε^{-\frac{2-p}{p-1}}$, respectively.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。