arXiv:2604.11729math.PRcs.DS2026-04

揭示了随机与确定性矩阵上一阶算法的普适性规律

Universality of first-order methods on random and deterministic matrices

论文配图:Universality of first-order methods on random and deterministic matrices
图 1 · 摘自论文原文
  • 通过交通分布分析,给出沃尔什-哈达玛德等矩阵的极限动态
  • 新算法在多种输入下保持高斯特性,统一了多个已有变体
  • 为算法设计提供新视角,适合研究优化理论与统计物理交叉者

一般一阶方法(GFOM)是一类通过矩阵-向量乘法和逐元素非线性更新状态向量的迭代算法。长期研究聚焦于‘高度随机’输入矩阵及近似消息传递(AMP)这一特殊情况,其状态渐近服从高斯分布。然而,如何构造对更结构化输入仍保持高斯性的算法,或解释现有AMP算法为何在某些确定性矩阵上表现良好,仍是未解之谜。本文通过输入矩阵的极限交通分布(traffic distribution)——即所有置换不变多项式极限值的集合——分析GFOM的图论展开,取得以下成果:1. 计算了首个非平凡确定性矩阵(包括沃尔什-哈达玛德、离散正弦/余弦变换矩阵的微小变体)的交通分布,从而确定了这些输入上的极限动态,解决了Marinari、Parisi和Ritort(1994)提出的部分长期猜想。2. 设计了一种新的AMP迭代,统一多个先前变体并推广至新输入类型,其极限动态在某些潜变量条件下为高斯分布。该分析适用于一大类自然交通分布(涵盖随机与确定性矩阵),且为最近Wang、Zhong和Fan(2022)提出的问题提供了简单的组合解释,揭示了Onsager修正项的本质。

原文摘要 · Abstract (English)

General first-order methods (GFOM) are a flexible class of iterative algorithms which update a state vector by matrix-vector multiplications and entrywise nonlinearities. A long line of work has sought to understand the large-n dynamics of GFOM, mostly focusing on "very random" input matrices and the approximate message passing (AMP) special case of GFOM whose state is asymptotically Gaussian. Yet, it has long remained unknown how to construct iterative algorithms that retain this Gaussianity for more structured inputs, or why existing AMP algorithms can be as effective for some deterministic matrices as they are for random matrices. We analyze diagrammatic expansions of GFOM via the limiting traffic distribution of the input matrix, the collection of all limiting values of permutation-invariant polynomials in the matrix entries, to obtain the following results: 1. We calculate the traffic distribution for the first non-trivial deterministic matrices, including (minor variants of) the Walsh-Hadamard and discrete sine and cosine transform matrices. This determines the limiting dynamics of GFOM on these inputs, resolving parts of longstanding conjectures of Marinari, Parisi, and Ritort (1994). 2. We design a new AMP iteration which unifies several previous AMP variants and generalizes to new input types, whose limiting dynamics are Gaussian conditional on some latent random variables. The asymptotic dynamics hold for a large and natural class of traffic distributions (encompassing both random and deterministic input matrices) and the algorithm's analysis gives a simple combinatorial interpretation of the Onsager correction, answering questions posed recently by Wang, Zhong, and Fan (2022).

优化算法统计物理矩阵分析高斯逼近

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。