提出流式数据下高效计算分布距离的新方法,适用于公平性与隐私审计。
Sublinear Algorithms for Wasserstein and Total Variation Distances: Applications to Fairness and Privacy Auditing
- 通过重构成频次估计问题,实现亚线性空间的分布函数学习
- 支持连续分布、无限支撑集,可实时估算Wasserstein与TV距离
- 在真实与合成数据上验证效率,适合算法公平性与隐私审计
在仅能访问样本的情况下,资源高效地计算概率分布及其间距离是数学科学中的基本且重要问题。本文提出一个通用框架,用于在样本以流形式到达时学习子韦布尔分布(即几乎任意轻尾或重尾分布)的概率密度函数(PDF)与累积分布函数(CDF)。核心思想是将问题转化为对某个适当选择的支撑集子集频率的估计,并在此基础上构建可合并的分布摘要。该方法仅需相对于观测样本数量的亚线性空间,即可实现从多个来源流入的样本中实时估计任意两分布之间的Wasserstein与总变差(TV)距离。相比现有方法存在超线性时间与线性空间复杂度的问题,本方法显著提升效率,并将可合并摘要框架扩展至具有可能无限支撑集的连续分布。结果在有界离散分布情形下达到现有下界,具有紧致性。此外,我们利用所提出的Wasserstein与TV距离估计器,严格审计了算法的公平性与隐私性。在合成与真实世界数据集上,实验充分验证了算法的高效性。
原文摘要 · Abstract (English)
Resource-efficiently computing representations of probability distributions and the distances between them while only having access to the samples is a fundamental and useful problem across mathematical sciences. In this paper, we propose a generic framework to learn the probability and cumulative distribution functions (PDFs and CDFs) of a sub-Weibull, i.e. almost any light- or heavy-tailed, distribution while the samples from it arrive in a stream. The idea is to reduce these problems into estimating the frequency of an \textit{appropriately chosen subset} of the support of a \textit{properly discretised distribution}. We leverage this reduction to compute mergeable summaries of distributions from the stream of samples while requiring only sublinear space relative to the number of observed samples. This allows us to estimate Wasserstein and Total Variation (TV) distances between any two distributions while samples arrive in streams and from multiple sources. Our algorithms significantly improves on the existing methods for distance estimation incurring super-linear time and linear space complexities, and further extend the mergeable summaries framework to continuous distributions with possibly infinite support. Our results are tight with respect to the existing lower bounds for bounded discrete distributions. In addition, we leverage our proposed estimators of Wasserstein and TV distances to tightly audit the fairness and privacy of algorithms. We empirically demonstrate the efficiency of proposed algorithms across synthetic and real-world datasets.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。