用探索性思维训练模型，减少性别偏见。
Mitigating Gender Bias via Fostering Exploratory Thinking in LLMs
- 让模型生成男女主角对等的情境故事并对比道德判断。
- 通过不一致判断引导模型生成中立评价，再微调模型。
- 既降偏见又不损模型能力，适合安全可控的AI应用。
大型语言模型常表现出性别偏见，导致在不同情境下对男性和女性主体的不平等对待。为此，我们提出一种新型数据生成框架，促使大模型进行探索性思考。该方法通过提示模型生成结构相同、道德模糊的男女主角故事对，并引发和比较其道德判断。当发现判断不一致时，引导模型生成平衡、性别中立的判断。这些故事-判断对用于通过直接偏好优化（DPO）对模型进行微调或优化。实验结果表明，该方法显著降低了性别偏见，同时保持甚至提升了模型的一般能力。代码与生成数据将公开：https://github.com/WeiKangda/LLMs-Exploratory-Bias-Mitigation/tree/main。
原文摘要 · Abstract (English)
Large Language Models (LLMs) often exhibit gender bias, resulting in unequal treatment of male and female subjects across different contexts. To address this issue, we propose a novel data generation framework that fosters exploratory thinking in LLMs. Our approach prompts models to generate story pairs featuring male and female protagonists in structurally identical, morally ambiguous scenarios, then elicits and compares their moral judgments. When inconsistencies arise, the model is guided to produce balanced, gender-neutral judgments. These story-judgment pairs are used to fine-tune or optimize the models via Direct Preference Optimization (DPO). Experimental results show that our method significantly reduces gender bias while preserving or even enhancing general model capabilities. We will release the code and generated data. We release the code and generated data at: https://github.com/WeiKangda/LLMs-Exploratory-Bias-Mitigation/tree/main.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。