证明最优控制变量可达到与最大似然估计相同的渐近方差。
It's all In the (Exponential) Family: An Equivalence between Maximum Likelihood Estimation and Control Variates for Sketching Algorithms
- 在指数族条件下，最优控制变量与最大似然估计等效。
- 该算法比传统求根方法更快且数值更稳定。
- 适用于需重复性与已知控制变量权重的机器学习场景。
最大似然估计（MLE）与控制变量估计（CVE）常结合用于机器学习中的压缩算法。本文证明，在特定指数族条件下，最优CVE的渐近方差与MLE相同，从而导出一种固定点算法求解MLE。实验表明，该算法在二元正态分布下比其他求根方法更快且数值更稳定，预计该优势在满足条件的分布中普遍成立。该算法还提升了使用MLE/CVE算法的可复现性，并可在控制变量权重已知时直接求得MLE。
原文摘要 · Abstract (English)
Maximum likelihood estimators (MLE) and control variate estimators (CVE) have been used in conjunction with known information across sketching algorithms and applications in machine learning. We prove that under certain conditions in an exponential family, an optimal CVE will achieve the same asymptotic variance as the MLE, giving a fixed point algorithm for the MLE. Experiments show the fixed point algorithm is faster and numerically stable compared to other root finding algorithms for the MLE for the bivariate Normal distribution, and we expect this to hold across distributions satisfying these conditions. We show how this algorithm leads to reproducibility for algorithms using MLE / CVE, and demonstrate how the algorithm leads to finding the MLE when the CV weights are known.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。