通过三元组对比学习，有效消除预训练模型中的社会偏见。
TriCon-Fair: Triplet Contrastive Learning for Mitigating Social Bias in Pre-trained Language Models
- 设计三元组对比损失，显式区分有偏负样本与无偏正样本。
- 在多个数据集上减少歧视性输出，同时保持下游任务性能。
- 适合需要公平性保障的敏感NLP应用，如招聘、信贷评审。
大规模语言模型的应用日益广泛，但其可能传播社会偏见，导致有害且不公平的结果。现有去偏方法独立处理有偏与无偏样本，忽略了二者间的相互关系，导致隐性的负-正耦合：对某一群体的改进可能损害另一群体，使残余偏见持续存在。本文提出TriCon-Fair，一种结合三元组对比与语言建模目标的对比学习框架，采用解耦损失函数。该方法为每个锚点样本显式分配一个有偏负样本和一个无偏正样本，解耦了正负样本的推拉动态，避免负-正耦合，并联合优化语言建模目标以保留通用能力。实验表明，TriCon-Fair在多个基准上均优于现有去偏基线，显著降低歧视性输出，同时维持强下游性能。结果表明，该方法为敏感NLP应用提供了可行且符合伦理的解决方案。
原文摘要 · Abstract (English)
The increasing utilization of large language models raises significant concerns about the propagation of social biases, which may result in harmful and unfair outcomes. However, existing debiasing methods treat the biased and unbiased samples independently, thus ignoring their mutual relationship. This oversight enables a hidden negative-positive coupling, where improvements for one group inadvertently compromise the other, allowing residual social bias to persist. In this paper, we introduce TriCon-Fair, a contrastive learning framework that employs a decoupled loss that combines triplet and language modeling terms to eliminate positive-negative coupling. Our TriCon-Fair assigns each anchor an explicitly biased negative and an unbiased positive, decoupling the push-pull dynamics and avoiding positive-negative coupling, and jointly optimizes a language modeling (LM) objective to preserve general capability. Experimental results demonstrate that TriCon-Fair reduces discriminatory output beyond existing debiasing baselines while maintaining strong downstream performance. This suggests that our proposed TriCon-Fair offers a practical and ethical solution for sensitive NLP applications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。