提出一种抗噪的广义矩估计方法,有效缓解异常值影响。
Robust Moment-Based Estimation via Spectral Gradient Reweighting
- 用谱梯度重加权机制软性调整每条数据的梯度贡献
- 在有限样本下实现参数估计误差与污染比例的显式依赖关系
- 适合处理含强噪声和异常值的低秩高斯混合模型估计
基于矩的估计是一种理论上有吸引力的参数推断方法,尤其在似然估计不可用、错误设定或计算不便时。然而,矩方程依赖样本均值,对异常值敏感。本文提出SGR-GMM算法,通过谱梯度重加权(SGR)机制,在矩匹配优化中软性重加权每条观测的梯度。分析包含三部分:第一,固定中心下,SGR被形式化为熵正则化的谱博弈,使用经典乘法权重和矩阵乘法权重后悔界进行分析;第二,建立了固定中心更新的显式收敛半径与有限终止界;第三,证明了局部有限样本参数估计误差界,明确依赖于污染比例、内点梯度稳定性、局部GMM可识别强度及优化精度。进一步将SGR-GMM特化为鲁棒对角加权GMM(DGMM),用于估计受加性高斯噪声和强污染影响的异方差低秩高斯混合模型。数值实验表明,SGR机制近似达到最优梯度估计,鲁棒DGMM显著优于非鲁棒基线。代码与数据见https://github.com/liu-lzhang/sgr-gmm。
原文摘要 · Abstract (English)
Moment-based estimation is a theoretically attractive approach to parametric inference, especially when likelihood-based estimation is unavailable, misspecified, or computationally inconvenient. However, the moment equations involve sample averages, which makes moment-based estimation sensitive to outliers. We propose the SGR-GMM algorithm, a robust generalized method of moments (GMM) procedure that uses a spectral gradient reweighting (SGR) primitive to soft-reweight the per-observation gradients during the moment-matching optimization. Our analysis has three layers. First, for a fixed center, the SGR primitive is formulated as an entropy-regularized spectral game between a sample-weight player and a density-matrix player, which is analyzed using classical multiplicative-weights and matrix-multiplicative-weights regret bounds. Second, we establish explicit convergence radius and finite termination bound for the fixed-center updates in the SGR primitive. Third, we prove a local finite-sample parameter estimation error bound with explicit dependence on the contamination fraction, inlier gradient stability, local GMM identification strength, and optimization accuracy. We further specialize the SGR-GMM algorithm to obtain a robust diagonally-weighted GMM (DGMM) estimator for estimating heteroscedastic low-rank Gaussian mixtures observed under additive Gaussian noise and strong contamination. In the numerical experiments, the SGR primitive produces nearly-oracle gradient estimation and the robust DGMM specialization substantially improves over non-robust moment baselines. The code and data are available at https://github.com/liu-lzhang/sgr-gmm.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。