通过激活模式精调，减少大模型误拒正常请求的问题。
Just Enough Shifts: Mitigating Over-Refusal in Aligned Language Models with Targeted Representation Fine-Tuning
- 基于内部激活模式定位拒答触发成分，精准调整
- 仅微调一层模型，多基准下显著降低误拒率
- 适合需要提升用户体验的对齐模型部署场景
安全对齐对大型语言模型抵御恶意指令至关重要，但常导致过度拒答，即错误拒绝正常请求，损害用户体验和模型实用性。我们提出ACTOR（基于激活的过拒减少训练），一种高效且计算与数据成本低的训练框架，利用多样化查询的内部激活模式，精准识别并调节引发拒答的激活成分，实现对拒答机制的更强控制。通过仅微调单个模型层，ACTOR在多个基准上有效降低过拒现象，同时保持处理有害请求的能力并维持整体模型性能。
原文摘要 · Abstract (English)
Safety alignment is crucial for large language models (LLMs) to resist malicious instructions but often results in over-refusals, where benign prompts are unnecessarily rejected, impairing user experience and model utility. We introduce ACTOR (Activation-Based Training for Over-Refusal Reduction), a robust and compute- and data-efficient training framework that minimizes over-refusals by leveraging internal activation patterns from diverse queries. ACTOR precisely identifies and adjusts the activation components that trigger refusals, providing stronger control over the refusal mechanism. By fine-tuning only a single model layer, ACTOR effectively reduces over-refusals across multiple benchmarks while maintaining the model's ability to handle harmful queries and preserve overall utility.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。