让大模型拒绝更灵活，避免机械回复‘我不能’
Beyond "I cannot fulfill this request": Alleviating Rigid Rejection in LLMs via Label Enhancement
- 用变分推断增强拒绝标签，生成细粒度拒绝分布
- 使模型能精准化解有害内容，不拒而安
- 适合追求自然对话体验的AI应用开发者
大型语言模型依赖安全对齐机制，在遵守安全请求的同时拒绝有害请求。然而，传统的拒绝机制常导致“僵化拒绝”，即使用统一模板（如‘我不能完成此请求’）无差别触发拒绝，严重损害人机交互的自然性。为此，本文提出LANCE，通过标签增强实现安全且灵活自然的响应。具体而言，LANCE采用变分推断进行标签增强，预测多个拒绝类别上的连续分布。这些细粒度的拒绝分布为精炼模型提供多向文本梯度，以中和提示中的危险元素，使大模型在避免僵化拒绝的同时生成安全回应，保持交互自然性。实验表明，LANCE显著缓解了僵化拒绝问题，同时维持高安全标准，在帮助性与自然性上显著优于现有基线模型。
原文摘要 · Abstract (English)
Large Language Models (LLMs) rely on safety alignment to obey safe requests while refusing harmful ones. However, traditional refusal mechanisms often lead to "rigid rejection," where a general template (e.g., "I cannot fulfill this request") indiscriminately triggers refusals and severely undermines the naturalness of interactions between humans and LLMs. To address this issue, LANCE is proposed in this paper to ensure safe yet flexible and natural responses via label enhancement. Specifically, LANCE employs variational inference to perform label enhancement, predicting a continuous distribution across multiple rejection categories. These fine-grained rejection distributions provide multi-way textual gradients for a refinement model to neutralize the hazardous elements in the prompt, so that the LLMs could generate safe responses that avoid rigid rejections while preserving the naturalness of interactions. Experiments demonstrate that LANCE significantly alleviates the rigid rejection problem while maintaining high security standards, significantly outperforming existing baseline models in terms of helpfulness and naturalness of responses.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。