arXiv:2602.04408cs.LGstat.ML2026-02

用信息论方法刻画公平性与性能的权衡边界,实现更可靠的模型公平性约束。

Separation-Utility Pareto Frontier: An Information-Theoretic Characterization

  • 从信息论角度刻画预测性能与敏感属性独立性的最优权衡关系
  • 提出基于条件互信息的正则化项,有效减少公平性违规且不牺牲性能
  • 适用于各类深度模型,适合需要可证明公平性的实际部署场景

本文从信息论视角研究预测性能与分离性(即预测结果在真实标签条件下应与敏感属性无关)之间的帕累托前沿。证明了该前沿的凹性,揭示了提升分离性所需边际性能代价递增的本质。进一步明确了严格权衡成立的条件,为实践中的权衡选择提供指导。基于理论分析,提出一种基于条件互信息(CMI)的可扩展正则化器,兼容梯度优化的任意深度模型,可作为训练过程中残留分离违规的标量监控指标,提供可计算的保障。数值实验在COMPAS、UCI Adult、UCI Bank和CelebA数据集上验证:所提方法显著降低分离性违规,同时保持或超越已有基线方法的性能。本研究提供了可证明、稳定且灵活的深度学习公平性约束方案。

原文摘要 · Abstract (English)

We study the Pareto frontier (optimal trade-off) between utility and separation, a fairness criterion requiring predictive independence from sensitive attributes conditional on the true outcome. Through an information-theoretic lens, we prove a characterization of the utility-separation Pareto frontier, establish its concavity, and thereby prove the increasing marginal cost of separation in terms of utility. In addition, we characterize the conditions under which this trade-off becomes strict, providing a guide for trade-off selection in practice. Based on the theoretical characterization, we develop an empirical regularizer based on conditional mutual information (CMI) between predictions and sensitive attributes given the true outcome. The CMI regularizer is compatible with any deep model trained via gradient-based optimization and serves as a scalar monitor of residual separation violations, offering tractable guarantees during training. Finally, numerical experiments support our theoretical findings: across COMPAS, UCI Adult, UCI Bank, and CelebA, the proposed method substantially reduces separation violations while matching or exceeding the utility of established baseline methods. This study thus offers a provable, stable, and flexible approach to enforcing separation in deep learning.

模型公平性信息论深度学习正则化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。