针对复杂相关数据,提出高效且最优的参数估计方法。
Optimal Estimation in Orthogonally Invariant Generalized Linear Models: Spectral Initialization and Approximate Message Passing
- 基于谱初始化与近似消息传递算法,实现最优估计
- 在任意奇异值分布下达到理论最优样本复杂度和误差
- 适用于真实数据中常见复杂相关结构,理论与实验一致
我们研究在正交不变随机设计矩阵下的广义线性模型中的参数估计问题。该模型允许设计矩阵具有任意奇异值分布,仅假设其奇异向量为通用型,是传统独立同分布高斯设计的极大推广,因真实数据常具有复杂相关结构,而后者方法可能严重失效。本文基于谱初始化迭代优化范式,提出最优谱估计器,并将其与近似消息传递(AMP)算法结合,严格证明了两步算法的性能保证。谱初始化和后续的AMP均达到了现有对估计极限的猜想——前者实现高效弱恢复的最优样本复杂度,后者达到最优误差。数值实验表明方法在非正交不变数据上仍有效,理论预测准确。
原文摘要 · Abstract (English)
We consider the problem of parameter estimation from a generalized linear model with a random design matrix that is orthogonally invariant in law. Such a model allows the design have an arbitrary distribution of singular values and only assumes that its singular vectors are generic. It is a vast generalization of the i.i.d. Gaussian design typically considered in the theoretical literature, and is motivated by the fact that real data often have a complex correlation structure so that methods relying on i.i.d. assumptions can be highly suboptimal. Building on the paradigm of spectrally-initialized iterative optimization, this paper proposes optimal spectral estimators and combines them with an approximate message passing (AMP) algorithm, establishing rigorous performance guarantees for these two algorithmic steps. Both the spectral initialization and the subsequent AMP meet existing conjectures on the fundamental limits to estimation -- the former on the optimal sample complexity for efficient weak recovery, and the latter on the optimal errors. Numerical experiments suggest the effectiveness of our methods and accuracy of our theory beyond orthogonally invariant data.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。