发现提示工程去偏效果虚假，模型常误判无偏内容为有偏。
Rethinking Prompt-based Debiasing in Large Language Models
- 用BBQ和StereoSet基准测试验证提示去偏有效性
- Llama2-7B-Chat误将超90%无偏内容判为有偏
- 提醒警惕评估指标陷阱，适合关注AI公平性的研究者
研究大型语言模型中的偏见问题对构建可信AI至关重要。尽管提示工程是常见方法，其有效性依赖于模型本身理解偏见的假设。本文通过在开源模型及商用GPT模型上使用BBQ和StereoSet基准进行系统分析，发现提示去偏往往流于表面：例如，Llama2-7B-Chat模型在无偏内容上错误识别率超过90%，尽管其在BBQ数据集上表现准确。此外，偏见评估任务中的特定提问方式常诱导模型给出回避性回答，忽略问题核心与上下文相关性。先前方法的成功可能源于评估指标缺陷。本研究揭示提示去偏存在‘虚假繁荣’，强调需重新审视偏见评估标准以实现真正可信的AI。
原文摘要 · Abstract (English)
Investigating bias in large language models (LLMs) is crucial for developing trustworthy AI. While prompt-based through prompt engineering is common, its effectiveness relies on the assumption that models inherently understand biases. Our study systematically analyzed this assumption using the BBQ and StereoSet benchmarks on both open-source models as well as commercial GPT model. Experimental results indicate that prompt-based is often superficial; for instance, the Llama2-7B-Chat model misclassified over 90% of unbiased content as biased, despite achieving high accuracy in identifying bias issues on the BBQ dataset. Additionally, specific evaluation and question settings in bias benchmarks often lead LLMs to choose "evasive answers", disregarding the core of the question and the relevance of the response to the context. Moreover, the apparent success of previous methods may stem from flawed evaluation metrics. Our research highlights a potential "false prosperity" in prompt-base efforts and emphasizes the need to rethink bias metrics to ensure truly trustworthy AI.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。