arXiv:2412.13003cs.LGstat.ML2024-12被引 1

用重要性采样提升子群体模型表现,兼顾平均与最差组性能。

Boosting Test Performance with Importance Sampling--a Subpopulation Perspective

  • 基于重要性采样构建统一框架,揭示子群体问题本质
  • 理论证明现有方法差异源于不同假设,解释平均性能下降原因
  • 仅一个估计器即可适配已知/未知属性场景,实测达到顶尖效果

尽管经验风险最小化(ERM)在机器学习中广泛应用,但在存在伪相关或由隐含属性引入的子群体时性能受限。现有方法虽能提升组平衡或最差组准确率,但以降低平均准确率为代价。多数工作对子群体方法进行孤立研究,未揭示其内在联系,阻碍技术进展。本文将重要性采样识别为解决子群体问题的简单而强大工具。理论上,我们提出子群体问题的新系统化表述,明确指出以往工作中未阐明的假设,揭示平均准确率下降的根本原因,并首次分析现有方法间的理论关联,识别出核心差异。应用上,我们证明单一估计器足以应对子群体问题,分别在属性已知和未知场景下设计实用方案。实验表明,在常用基准数据集上达到当前最优性能。

原文摘要 · Abstract (English)

Despite empirical risk minimization (ERM) is widely applied in the machine learning community, its performance is limited on data with spurious correlation or subpopulation that is introduced by hidden attributes. Existing literature proposed techniques to maximize group-balanced or worst-group accuracy when such correlation presents, yet, at the cost of lower average accuracy. In addition, many existing works conduct surveys on different subpopulation methods without revealing the inherent connection between these methods, which could hinder the technology advancement in this area. In this paper, we identify important sampling as a simple yet powerful tool for solving the subpopulation problem. On the theory side, we provide a new systematic formulation of the subpopulation problem and explicitly identify the assumptions that are not clearly stated in the existing works. This helps to uncover the cause of the dropped average accuracy. We provide the first theoretical discussion on the connections of existing methods, revealing the core components that make them different. On the application side, we demonstrate a single estimator is enough to solve the subpopulation problem. In particular, we introduce the estimator in both attribute-known and -unknown scenarios in the subpopulation setup, offering flexibility in practical use cases. And empirically, we achieve state-of-the-art performance on commonly used benchmark datasets.

子群体重要性采样公平性

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。