探究表格大模型的因果预训练如何影响公平性，发现其提升有限且不稳定。
Causal Pre-training Under the Fairness Lens: An Empirical Study of TabPFN
- 用结构因果模型生成海量合成数据进行因果预训练
- 在分布偏移下预测准确率高但公平性改善不显著
- 特别在缺失非随机场景下公平性表现不佳，适合关注公平性的研究者
用于表格数据的基础模型（如TabPFN）通过结构因果模型（SCM）生成的大量合成数据进行预训练，利用上下文学习在真实任务中实现高预测精度。然而，这类模型在预训练中融入因果推理的思想，其公平性特性尚未被充分研究。本文对TabPFN及其微调变体进行了全面的实证评估，考察了不同数据规模和分布偏移下的预测性能、公平性和鲁棒性。结果表明，尽管TabPFN相比基线模型具备更强的预测能力并能抵抗虚假相关，但在公平性方面的改进程度有限且不一致，尤其在缺失非随机（MNAR）协变量偏移下表现更差。这说明当前因果预训练虽有帮助但不足以保障算法公平性，提示在实际部署类似模型时需引入额外公平性干预。
原文摘要 · Abstract (English)
Foundation models for tabular data, such as the Tabular Prior-data Fitted Network (TabPFN), are pre-trained on a massive number of synthetic datasets generated by structural causal models (SCM). They leverage in-context learning to offer high predictive accuracy in real-world tasks. However, the fairness properties of these foundational models, which incorporate ideas from causal reasoning during pre-training, remain underexplored. In this work, we conduct a comprehensive empirical evaluation of TabPFN and its fine-tuned variants, assessing predictive performance, fairness, and robustness across varying dataset sizes and distributional shifts. Our results reveal that while TabPFN achieves stronger predictive accuracy compared to baselines and exhibits robustness to spurious correlations, improvements in fairness are moderate and inconsistent, particularly under missing-not-at-random (MNAR) covariate shifts. These findings suggest that the causal pre-training in TabPFN is helpful but insufficient for algorithmic fairness, highlighting implications for deploying TabPFN (and similar) models in practice and the need for further fairness interventions.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。