提出通用去偏框架，让语言模型生成表格数据时更公平
Towards Universal Debiasing for Language Models-based Tabular Data Generation
- 通过降低优势属性与受保护属性间的互信息实现去偏
- 在多个数据集上提升公平性，同时保持数据生成质量
- 无需微调模型即可去偏，适合高风险场景应用
大语言模型（LLMs）在表格数据生成中表现优异，但其训练数据中的历史偏差常导致模型加剧不公平问题，尤其当涉及多个优势属性和受保护属性时。本文提出一种通用去偏框架，通过同时减少优势属性与受保护属性之间的组级依赖关系来缓解该问题。利用基于LLM的表格生成器的自回归结构和解析采样分布，该方法高效计算互信息，避免了复杂的数值估计。在此基础上，提出两种互补策略：一种是与现有模型无缝集成的直接偏好优化（DPO）方法（UDF-DPO），另一种是无需调整模型参数即可实现去偏的针对性技术（UDF-MIX）。大量实验表明，该框架在公平性与数据实用性之间取得良好平衡，为高风险应用提供可扩展、实用的去偏解决方案。
原文摘要 · Abstract (English)
Large language models (LLMs) have achieved promising results in tabular data generation. However, inherent historical biases in tabular datasets often cause LLMs to exacerbate fairness issues, particularly when multiple advantaged and protected features are involved. In this work, we introduce a universal debiasing framework that minimizes group-level dependencies by simultaneously reducing the mutual information between advantaged and protected attributes. By leveraging the autoregressive structure and analytic sampling distributions of LLM-based tabular data generators, our approach efficiently computes mutual information, reducing the need for cumbersome numerical estimations. Building on this foundation, we propose two complementary methods: a direct preference optimization (DPO)-based strategy, namely UDF-DPO, that integrates seamlessly with existing models, and a targeted debiasing technique, namely UDF-MIX, that achieves debiasing without tuning the parameters of LLMs. Extensive experiments demonstrate that our framework effectively balances fairness and utility, offering a scalable and practical solution for debiasing in high-stakes applications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。