研究模型更新下的公平性审计,揭示可信赖的审计边界。
Auditing Fairness under Model Updates: Fundamental Complexity and Property-Preserving Updates
- 提出基于经验属性优化的通用审计框架
- 定义SP维度刻画更新复杂度,实现无分布审计界
- 适用于公平性、预测误差等多类审计目标
随着机器学习模型深度嵌入社会基础设施,对其偏见进行审计愈发重要。然而在实际部署中,模型所有者可能根据环境变化(如金融市场)自适应更新模型,这些更新会改变模型类别但保持某些关注属性不变,引发根本性问题:在模型迁移下哪些属性仍可可靠审计?本文研究任意更新下的群体公平性审计,考虑一类广义转移——在改变审计前模型类别的同时保持被审计属性的不变性。目标有二:(i)刻画允许更新的信息复杂度,识别哪些策略性变更可保持审计属性;(ii)以最少标注样本高效估计审计属性(如群体公平性)。提出基于经验属性优化(EPO)oracle的PAC审计通用框架。针对统计均等性,建立由新型组合度量SP维度表征的无分布审计边界。最后证明该框架可自然扩展至预测误差与鲁棒风险等其他审计目标。
原文摘要 · Abstract (English)
As machine learning models become increasingly embedded in societal infrastructure, auditing them for bias is of growing importance. However, in real-world deployments, auditing is complicated by the fact that model owners may adaptively update their models in response to changing environments, such as financial markets. These updates can alter the underlying model class while preserving certain properties of interest, raising fundamental questions about what can be reliably audited under such shifts. In this work, we study group fairness auditing under arbitrary updates. We consider general shifts that modify the pre-audit model class while maintaining invariance of the audited property. Our goals are two-fold: (i) to characterize the information complexity of allowable updates, by identifying which strategic changes preserve the property under audit; and (ii) to efficiently estimate auditing properties, such as group fairness, using a minimal number of labeled samples. We propose a generic framework for PAC auditing based on an Empirical Property Optimization (EPO) oracle. For statistical parity, we establish distribution-free auditing bounds characterized by the SP dimension, a novel combinatorial measure that captures the complexity of admissible strategic updates. Finally, we demonstrate that our framework naturally extends to other auditing objectives, including prediction error and robust risk.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。