arXiv:2410.20760stat.MLcs.LG2024-10被引 1

提出新型距离度量,提升核指数族模型的鲁棒估计性能。

Robust Estimation for Kernel Exponential Families with Smoothed Total Variation Distances

  • 用平滑总变差距离构建对抗性估计器,适配广义统计模型。
  • 理论证明该方法对分布污染具有鲁棒性,误差受控于样本量。
  • 适用于复杂生成模型中的高维或无限维核指数族,适合数据含异常值场景。

在统计推断中,通常假设样本独立同分布于预设统计模型。然而实际中这一假设常被破坏,极端样本(异常值)会显著影响经典估计器。鲁棒统计研究如何在理想假设不成立时仍保持可靠的方法。近期研究发现,如Tukey中位数等鲁棒估计器可被生成对抗网络(GAN)近似,而GAN基于积分概率度量(IPM)。但现有理论分析多限于高斯或椭球分布模型。本文将GAN类估计器推广至更一般的统计模型——核指数族,涵盖有限与无限维情形。为此,我们提出平滑总变差(STV)距离作为一类IPM,并理论分析其构造的估计器在核指数族下的鲁棒性。结果表明,该估计器对分布污染具有鲁棒性。此外,我们分析了蒙特卡洛近似方法在处理归一化常数计算难题时的预测精度,为实际应用提供支持。

原文摘要 · Abstract (English)

In statistical inference, we commonly assume that samples are independent and identically distributed from a probability distribution included in a pre-specified statistical model. However, such an assumption is often violated in practice. Even an unexpected extreme sample called an {\it outlier} can significantly impact classical estimators. Robust statistics studies how to construct reliable statistical methods that efficiently work even when the ideal assumption is violated. Recently, some works revealed that robust estimators such as Tukey's median are well approximated by the generative adversarial net (GAN), a popular learning method for complex generative models using neural networks. GAN is regarded as a learning method using integral probability metrics (IPM), which is a discrepancy measure for probability distributions. In most theoretical analyses of Tukey's median and its GAN-based approximation, however, the Gaussian or elliptical distribution is assumed as the statistical model. In this paper, we explore the application of GAN-like estimators to a general class of statistical models. As the statistical model, we consider the kernel exponential family that includes both finite and infinite-dimensional models. To construct a robust estimator, we propose the smoothed total variation (STV) distance as a class of IPMs. Then, we theoretically investigate the robustness properties of the STV-based estimators. Our analysis reveals that the STV-based estimator is robust against the distribution contamination for the kernel exponential family. Furthermore, we analyze the prediction accuracy of a Monte Carlo approximation method, which circumvents the computational difficulty of the normalization constant.

鲁棒统计核指数族生成模型对抗学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。