通过对抗训练提升摘要模型鲁棒性，有效缓解姓名-国籍与政治倾向偏见。
AdvSumm: Adversarial Training for Bias Mitigation in Text Summarization
- 在序列到序列模型嵌入层引入梯度引导扰动，增强对输入变化的鲁棒性。
- 在多个基准数据集上显著降低姓名-国籍与政治表述偏见，不牺牲摘要质量。
- 适用于需公平输出的现实场景，如新闻摘要、政策报告生成。
大型语言模型在文本摘要任务中表现优异，但常继承预训练数据中的关联与框架偏见，导致下游任务输出不当或不公平。本文提出AdvSumm（对抗摘要），一种无需领域依赖的训练框架，通过提升泛化能力来缓解摘要偏见。受对抗鲁棒性启发，AdvSumm引入新型扰动器组件，在序列到序列模型的嵌入层施加梯度引导扰动，增强模型对输入变化的鲁棒性。实验证明，AdvSumm能有效降低姓名-国籍偏见与政治表述偏见，且不损害摘要质量。相比标准Transformer及回译等数据增强方法，AdvSumm在多个基准数据集上展现更强的偏见缓解效果。
原文摘要 · Abstract (English)
Large Language Models (LLMs) have achieved impressive performance in text summarization and are increasingly deployed in real-world applications. However, these systems often inherit associative and framing biases from pre-training data, leading to inappropriate or unfair outputs in downstream tasks. In this work, we present AdvSumm (Adversarial Summarization), a domain-agnostic training framework designed to mitigate bias in text summarization through improved generalization. Inspired by adversarial robustness, AdvSumm introduces a novel Perturber component that applies gradient-guided perturbations at the embedding level of Sequence-to-Sequence models, enhancing the model's robustness to input variations. We empirically demonstrate that AdvSumm effectively reduces different types of bias in summarization-specifically, name-nationality bias and political framing bias-without compromising summarization quality. Compared to standard transformers and data augmentation techniques like back-translation, AdvSumm achieves stronger bias mitigation performance across benchmark datasets.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。