自动生成伪有害提示，评估大模型误拒问题。
Automatic Pseudo-Harmful Prompt Generation for Evaluating False Refusals in Large Language Models
- 基于模型特性自动生成可控伪有害提示
- 构建十倍于现有数据集的PHTest，覆盖更多误拒模式
- 揭示安全与可用性间的权衡，适合模型开发者使用
对齐安全的大语言模型（LLMs）有时会错误拒绝伪有害提示，如“如何杀死蚊子”，这些提示实际无害。频繁误拒不仅让用户沮丧，还引发公众对对齐价值的反弹。本文提出首个自动生成多样化、内容可控且模型依赖的伪有害提示的方法。利用该方法，我们构建了名为PHTest的评估数据集，其规模是现有数据集的十倍，涵盖更多误拒模式，并对争议性提示进行单独标注。我们在PHTest上评估了20个LLM，揭示了减少误拒与提升对抗越狱攻击能力之间的权衡。此外，我们发现许多越狱防御措施显著提高了误拒率，从而损害可用性。本方法和数据集有助于开发者评估和微调更安全且更易用的LLM。代码与数据集已开源。
原文摘要 · Abstract (English)
Safety-aligned large language models (LLMs) sometimes falsely refuse pseudo-harmful prompts, like "how to kill a mosquito," which are actually harmless. Frequent false refusals not only frustrate users but also provoke a public backlash against the very values alignment seeks to protect. In this paper, we propose the first method to auto-generate diverse, content-controlled, and model-dependent pseudo-harmful prompts. Using this method, we construct an evaluation dataset called PHTest, which is ten times larger than existing datasets, covers more false refusal patterns, and separately labels controversial prompts. We evaluate 20 LLMs on PHTest, uncovering new insights due to its scale and labeling. Our findings reveal a trade-off between minimizing false refusals and improving safety against jailbreak attacks. Moreover, we show that many jailbreak defenses significantly increase the false refusal rates, thereby undermining usability. Our method and dataset can help developers evaluate and fine-tune safer and more usable LLMs. Our code and dataset are available at https://github.com/umd-huang-lab/FalseRefusal
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。