arXiv:2506.11994stat.MLcs.LG2025-06NeurIPS

从局部小矩阵推断超大矩阵的谱分布,解决无法直接计算的问题。

Spectral Estimation with Free Decompression

论文配图:Spectral Estimation with Free Decompression
图 1 · 摘自论文原文
  • 基于自由概率论,通过小块子矩阵反推整体谱分布。
  • 在合成数据和真实数据上均有效匹配全矩阵的特征谱。
  • 适用于分布式学习中无法获取完整矩阵的场景,适合算法研究者。

计算超大矩阵的特征值是机器学习中许多关键任务的核心,如对数行列式、矩阵函数迹等指标的评估。随着数据规模增长,协方差与核矩阵日益庞大,直接构造已不可行。现有方法依赖矩阵-向量乘积,但当数据分布于多节点或矩阵仅以隐式定义时,连完整矩阵向量乘积也无法获得,导致矩阵“不可触碰”。本文提出“自由去压缩”方法,利用自由概率理论,从极小子矩阵的实测谱密度出发,外推推断超大(不可触碰)矩阵的特征谱。通过合成数据验证其在随机矩阵理论极限分布下的表现,并应用于真实数据子矩阵,结果与全矩阵实测谱高度吻合。

原文摘要 · Abstract (English)

Computing eigenvalues of very large matrices is a critical task in many machine learning applications, including the evaluation of log-determinants, the trace of matrix functions, and other important metrics. As datasets continue to grow in scale, the corresponding covariance and kernel matrices become increasingly large, often reaching magnitudes that make their direct formation impractical or impossible. Existing techniques typically rely on matrix-vector products, which can provide efficient approximations, if the matrix spectrum behaves well. However, in settings like distributed learning, or when the matrix is defined only indirectly, access to the full data set can be restricted to only very small sub-matrices of the original matrix. In these cases, the matrix of nominal interest is not even available as an implicit operator, meaning that even matrix-vector products may not be available. In such settings, the matrix is "impalpable," in the sense that we have access to only masked snapshots of it. We draw on principles from free probability theory to introduce a novel method of "free decompression" to estimate the spectrum of such matrices. Our method can be used to extrapolate from the empirical spectral densities of small submatrices to infer the eigenspectrum of extremely large (impalpable) matrices (that we cannot form or even evaluate with full matrix-vector products). We demonstrate the effectiveness of this approach through a series of examples, comparing its performance against known limiting distributions from random matrix theory in synthetic settings, as well as applying it to submatrices of real-world datasets, matching them with their full empirical eigenspectra.

谱估计自由概率大规模矩阵去压缩

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。