让大模型既说得多又说对，通过识别未知知识和表达不确定来平衡信息量与真实性。
Balancing Truthfulness and Informativeness with Uncertainty-Aware Instruction Fine-Tuning
- 从指令微调数据中剔除陌生知识，减少模型生成错误信息
- 训练模型识别自身不确定性，输出时主动标注不确定内容
- 适合需要高可信度回答的场景，如医疗、法律咨询
指令微调（IFT）虽能提升大语言模型的信息密度，但会降低其真实性。这是因为微调引导模型生成预训练阶段未充分覆盖的长尾知识，导致在面对新任务时泛化能力下降，出现幻觉。本文实证揭示了微调数据中的陌生知识如何损害模型的真实性，并提出两种新范式：$UNIT_{cut}$ 通过识别并移除微调数据中的陌生知识以减轻负面影响；$UNIT_{ref}$ 则训练模型识别自身不确定性，在回答末尾明确标注不确定内容。实验表明，$UNIT_{cut}$ 显著提升模型真实性，$UNIT_{ref}$ 在保持高信息量的同时有效减少幻觉，区分自信与不确定陈述。
原文摘要 · Abstract (English)
Instruction fine-tuning (IFT) can increase the informativeness of large language models (LLMs), but may reduce their truthfulness. This trade-off arises because IFT steers LLMs to generate responses containing long-tail knowledge that was not well covered during pre-training. As a result, models become more informative but less accurate when generalizing to unseen tasks. In this paper, we empirically demonstrate how unfamiliar knowledge in IFT datasets can negatively affect the truthfulness of LLMs, and we introduce two new IFT paradigms, $UNIT_{cut}$ and $UNIT_{ref}$, to address this issue. $UNIT_{cut}$ identifies and removes unfamiliar knowledge from IFT datasets to mitigate its impact on model truthfulness, whereas $UNIT_{ref}$ trains LLMs to recognize their uncertainty and explicitly indicate it at the end of their responses. Our experiments show that $UNIT_{cut}$ substantially improves LLM truthfulness, while $UNIT_{ref}$ maintains high informativeness and reduces hallucinations by distinguishing between confident and uncertain statements.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。