提出用假设检验方法实现对数据分布的精准遗忘,兼顾性能与统计可靠性。
Statistical Unlearning of Distributions: A Hypothesis Testing Approach

- 基于假设检验设计可解释的删样策略,实现分布级遗忘
- 刻画了删样后数据分布的允许范围与性能权衡边界
- 适用于高维高斯、泊松等多类分布,适合需消除偏见或版权数据的场景
机器学习系统面临不仅删除单个数据点,还需消除整个信息域(如有害语言、版权语料、人口偏差)的需求。这引发统计-计算权衡困境:完全移除所有相关样本计算成本过高,随机删减又无法保证分布级统计效果。本文提出一种分布级遗忘的统计框架,将领域建模为概率分布,目标是选取特定样本子集进行删除,以削弱不想要分布的影响同时保持期望分布上的性能。通过在编辑后的数据上进行假设检验,建立可解释且鲁棒的删样选择准则。在该框架下,我们刻画了允许的编辑后分布区域及删样-保留帕累托前沿,涵盖参数族(如任意维度平移高斯、一维位置族带对数凹噪声、一维泊松族)和非参数族(如高斯白噪声模型,用于非参数回归)。我们证明了多模态不想要分布组合下的复合规则,并揭示当组合大量此类分布时,删样-保留基线呈现中心极限行为。最后,针对某些选择算法提供有限样本的帕累托前沿,观察到信息-计算间隙。
原文摘要 · Abstract (English)
Machine learning systems increasingly face requirements to forget not only individual data points, but entire domains of information, such as toxic language, copyrighted corpora, or demographic biases. This raises a fundamental dilemma of statistical-computational tradeoffs: removing all samples from an unwanted domain may be computationally prohibitive, while randomly removing a subset may not provide distribution-level statistical guarantees. We propose a statistical framework for distributional unlearning, in which domains are modeled as probability distributions, and the goal is to remove a carefully chosen subset of samples that reduces the effect of an unwanted distribution while preserving performance on a desired one. We formalize this using a hypothesis test of the edited data with the desired and unwanted domains, leading to an interpretable and robust criterion for selecting samples to remove. Within this statistical framework, we characterize the fundamental region of the allowable edited data distributions and the removal-preservation Pareto frontier for a broad class of distribution families. This includes parametric families such as shifted Gaussians of arbitrary dimension, a one-dimensional location family with log-concave noise, and the one-dimensional Poisson family. It also includes nonparametric families such as the Gaussian white noise model, a canonical model for nonparametric regression. We prove composition rules that describe how distributional unlearning behaves across multimodal unwanted domains, and introduce a central-limit behavior for the removal-preservation baselines when composing a large number of such families. Finally, we provide finite sample guarantees by providing Pareto frontiers for some selection algorithms, and observe an information-computation gap.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。