arXiv:2507.19290cs.DScs.LG2025-07被引 4

用更少的查询学习矩阵结构,速度提升近似平方根倍。

Query Efficient Structured Matrix Learning

  • 通过矩阵向量乘法查询,高效逼近任意有限矩阵族中的最优结构。
  • 将查询复杂度从 O(log|F|) 降低至 Õ(√log|F|),实现近似平方根级提升。
  • 适用于线性矩阵族等无限情形,适合科学计算与优化中的快速建模场景。

我们研究在仅能访问矩阵-向量乘法查询(x → Ax 与 x → Aᵀx)的情况下,学习未知矩阵 A 的结构化近似(如低秩、稀疏、带状等)的问题。该问题在科学计算和机器学习中具有核心意义,涉及快速矩阵乘法与求逆、构建优化预条件子及微分算子学习等应用。以往工作主要关注特定结构矩阵族的查询复杂度上下界。本文首次从更一般角度出发,旨在理解从任意有限矩阵族 𝒫 中学习近似解的查询复杂度。标准矩阵压缩理论表明,使用 O(log|𝒫|) 次矩阵-向量查询即可完成。该上界对已广泛研究的向量-矩阵-向量查询(x,y → xᵀAy)也是最优的。然而,令人惊讶的是,在矩阵-向量查询模型下,我们证明可实现近似二次提升,将复杂度降至 Õ(√log|𝒫|),且该结果在对数对数因子范围内紧致。通过覆盖数论证,该结论可推广至经典无限矩阵族。例如,对于任意维度为 q 的线性矩阵族,可仅用 Õ(√q) 次矩阵-向量查询获得近似最优逼近,优于传统基于压缩技术或向量-矩阵-向量查询的 O(q) 上界。

原文摘要 · Abstract (English)

We study the problem of learning a structured approximation (low-rank, sparse, banded, etc.) to an unknown matrix $A$ given access to matrix-vector product (matvec) queries of the form $x \rightarrow Ax$ and $x \rightarrow A^Tx$. This problem is of central importance to algorithms across scientific computing and machine learning, with applications to fast multiplication and inversion for structured matrices, building preconditioners for first-order optimization, and as a model for differential operator learning. Prior work focuses on obtaining query complexity upper and lower bounds for learning specific structured matrix families that commonly arise in applications. We initiate the study of the problem in greater generality, aiming to understand the query complexity of learning approximations from general matrix families. Our main result focuses on finding a near-optimal approximation to $A$ from any finite-sized family of matrices, $\mathcal{F}$. Standard results from matrix sketching show that $O(\log|\mathcal{F}|)$ matvec queries suffice in this setting. This bound can also be achieved, and is optimal, for vector-matrix-vector queries of the form $x,y\rightarrow x^TAy$, which have been widely studied in work on rank-$1$ matrix sensing. Surprisingly, we show that, in the matvec model, it is possible to obtain a nearly quadratic improvement in complexity, to $\tilde{O}(\sqrt{\log|\mathcal{F}|})$. Further, we prove that this bound is tight up to log-log factors. Via covering number arguments, our result extends to well-studied infinite families. As an example, we establish that a near-optimal approximation from any \emph{linear matrix family} of dimension $q$ can be learned with $\tilde{O}(\sqrt{q})$ matvec queries, improving on an $O(q)$ bound achievable via sketching techniques and vector-matrix-vector queries.

矩阵学习查询效率结构化矩阵优化算法

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。