提出无偏分箱方法,让数据分桶更公平,减少算法歧视。
Unbiased Binning for Fairness-aware Attribute Representation
- 从分桶边界候选集出发,用动态规划找最优分箱
- 引入ε容忍偏差,解决极端情况下无解问题
- 结合分治与局部搜索,高效处理大规模数据
将原始特征离散化为分桶表示是共享数据集的常见步骤,但该过程可能引发显著偏差并放大下游任务中的不公平性。本文提出无偏分箱问题:在给定属性下,寻找最接近等大小分箱、且满足不同分桶间群体平等的离散化方案。通过定义一组边界候选集,证明无偏分箱的边界必须来自该集合,并基于此开发了高效的动态规划算法。当群体分布差异大时,无解或公平代价过高,因此引入ε-有偏分箱问题,将组间差异限制在小值ε内。提出一种二次时间复杂度的动态规划解法(DP),但其不适用于超大规模场景。为此设计基于局部搜索(LS)的可扩展算法,核心是近线性时间的分治(D&C)子算法,能在存在解时找到有效解。随后以D&C结果为上界启动局部搜索,逼近最优解。
原文摘要 · Abstract (English)
Discretizing raw features into bucketized attribute representations is a popular step before sharing a dataset. It is, however, evident that this step can cause significant bias in data and amplify unfairness in downstream tasks. In this paper, we address this issue by introducing the unbiased binning problem that, given an attribute to bucketize, finds its closest discretization to equal-size binning that satisfies group parity across different buckets. Defining a small set of boundary candidates, we prove that unbiased binning must select its boundaries from this set. We then develop an efficient dynamic programming algorithm on top of the boundary candidates to solve the unbiased binning problem. Finding an unbiased binning may sometimes result in a high price of fairness, or it may not even exist, especially when group values follow different distributions. Considering that a small bias in the group ratios may be tolerable in such settings, we introduce the epsilon-biased binning problem that bounds the group disparities across buckets to a small value epsilon. We first develop a dynamic programming solution, DP, that finds the optimal binning in quadratic time. The DP algorithm, while polynomial, does not scale to very large settings. Therefore, we propose a practically scalable algorithm, based on local search (LS), for epsilon-biased binning. The key component of the LS algorithm is a divide-and-conquer (D&C) algorithm that finds a near-optimal solution for the problem in near-linear time. We prove that D&C finds a valid solution for the problem unless none exists. The LS algorithm then initiates a local search, using the D&C solution as the upper bound, to find the optimal solution.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。