通过梯度驱动优化，让大模型更准拒绝不懂的问题，减少幻觉同时不误拒可答问题。
GRAIT: Gradient-Driven Refusal-Aware Instruction Tuning for Effective Hallucination Mitigation
- 基于梯度选择难样本，精准识别并拒绝未知问题
- 自适应加权机制降低过度拒绝风险，保持回答有用性
- 适合需要高可靠性的问答场景，如医疗、法律
拒绝感知指令微调（RAIT）旨在通过提升大语言模型对超出知识范围问题的拒绝能力，减少幻觉并增强可靠性。有效实现RAIT需应对两大挑战：一是有效拒绝未知问题以最小化幻觉；二是避免过度拒绝，确保可回答的问题不被误拒，从而维持输出的实用性。本文从梯度视角出发，提出梯度驱动的拒绝感知指令微调框架GRAIT：（1）采用梯度驱动的样本选择策略，有效抑制幻觉；（2）在微调中引入自适应加权机制，降低过拒风险，实现准确拒绝与有用响应之间的平衡。在开放式和多选题问答任务上的实验表明，GRAIT显著优于现有RAIT方法，在整体性能上表现更优。源代码与数据将公开于https://github.com/opendatalab/GRAIT。
原文摘要 · Abstract (English)
Refusal-Aware Instruction Tuning (RAIT) aims to enhance Large Language Models (LLMs) by improving their ability to refuse responses to questions beyond their knowledge, thereby reducing hallucinations and improving reliability. Effective RAIT must address two key challenges: firstly, effectively reject unknown questions to minimize hallucinations; secondly, avoid over-refusal to ensure questions that can be correctly answered are not rejected, thereby maintain the helpfulness of LLM outputs. In this paper, we address the two challenges by deriving insightful observations from the gradient-based perspective, and proposing the Gradient-driven Refusal Aware Instruction Tuning Framework GRAIT: (1) employs gradient-driven sample selection to effectively minimize hallucinations and (2) introduces an adaptive weighting mechanism during fine-tuning to reduce the risk of over-refusal, achieving the balance between accurate refusals and maintaining useful responses. Experimental evaluations on open-ended and multiple-choice question answering tasks demonstrate that GRAIT significantly outperforms existing RAIT methods in the overall performance. The source code and data will be available at https://github.com/opendatalab/GRAIT .
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。