动态环境中自适应去偏,保持预测能力的同时提升公平性。
AdapFair: Ensuring Adaptive Fairness for Machine Learning Operations
- 用归一化流实现高效无损数据去偏转换
- 结合Wasserstein距离优化,确保公平性且不损失关键信息
- 无需重训练,适合频繁数据漂移的实时场景
机器学习算法的偏见与歧视问题备受关注,已有多种针对特定场景的解决方案,但难以应对机器学习运营中的内在公平性挑战。本文提出一种自适应去偏框架,旨在动态条件下找到最优公平的数据变换,最大程度保留数据预测能力。该方法具有灵活性与高效性,可集成于预训练黑箱分类器,即使面对频繁数据漂移、不断变化的公平性要求以及批量相似任务,也仅需极少重训练即可提供公平性保障。我们利用归一化流实现高效、信息保持型的数据变换,避免关键信息丢失;同时引入Wasserstein距离作为公平性度量,指导数据变换的优化。此外,设计了一种具有闭式梯度计算的高效优化算法,使框架具备可扩展性,适用于动态真实环境。
原文摘要 · Abstract (English)
The biases and discrimination of machine learning algorithms have attracted significant attention, leading to the development of various algorithms tailored to specific contexts. However, these solutions often fall short of addressing fairness issues inherent in machine learning operations. In this paper, we present an adaptive debiasing framework designed to find an optimal fair transformation of input data that maximally preserves data predictability under dynamic conditions. A distinctive feature of our approach is its flexibility and efficiency. It can be integrated with pretrained black-box classifiers, providing fairness guarantees with minimal retraining efforts, even in the face of frequent data drifts, evolving fairness requirements, and batches of similar tasks. To achieve this, we leverage the normalizing flows to enable efficient, information-preserving data transformation, ensuring that no critical information is lost during the debiasing process. Additionally, we incorporate the Wasserstein distance as the fairness measure to guide the optimization of data transformations. Finally, we introduce an efficient optimization algorithm with closed-formed gradient computations, making our framework scalable and suitable for dynamic, real-world environments.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。