用大模型自动分析数据公平性,发现潜在歧视并生成报告。
Automatic Causal Fairness Analysis with LLM-Generated Reporting

- 基于因果模型构建公平性分析框架,支持反事实推理。
- 零样本调用大模型生成报告,准确率优于直接分析。
- 支持连续目标与有序保护变量,适合实际应用落地。
AutoML旨在自动化机器学习在现实问题中的应用,是推动AI普及的关键步骤。然而,大多数AutoML框架未考虑训练数据和预测结果中可能存在的不公平性。我们提出 extsc{FairMind} 软件原型,旨在实现数据集层面的公平性自动化分析。该方法基于Plečko和Bareinboim近期提出的标准公平性模型,通过反事实查询对目标变量、潜在混杂因素、中介变量及受保护特征的不同取值进行因果效应评估。在完成必要数据预处理后,工具实现效应的闭式计算。随后利用大模型生成关于训练数据集中检测到的公平性水平的准确报告。我们在零样本设置下实现此目标,并通过示例展示其相较于直接由大模型进行分析的优势。为促进实际应用,还讨论了对有序受保护变量和连续目标的支持,以及新的分解结果。
原文摘要 · Abstract (English)
AutoML, intended as the process of automating the application of machine learning to real-world problems, is a key step for AI popularisation. Most AutoML frameworks are not accounting for the potential lack of fairness in the training data and in the corresponding predictions. We introduce \textsc{FairMind}, a software prototype aiming to automatise fairness analysis at the dataset level. We achieve that by resorting to the assumptions of the \emph{standard fairness model}, recently proposed by Plečko and Bareinboim. This allows for a sound fairness evaluation in terms of causal effects, based on \emph{counterfactual} queries involving the target, possibly confounders and mediators, and the different values of an input feature we regard as \emph{protected}. After the necessary data preprocessing, the tool implements a closed-form computation of the effects. LLMs are consequently exploited to generate accurate reports on the fairness levels detected in the training dataset. We achieve that in a zero-shot setup and show by examples the expected advantages with respect to a direct analysis performed by the LLM. To favour applications, extensions to ordinal protected variable and continuous targets and novel decomposition results are also discussed.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。