让大模型学会识别自己不知道的问答，提升可信度。
What Models Know, How Well They Know It: Knowledge-Weighted Fine-Tuning for Learning When to Say "I Don't Know"
- 通过多次采样推理计算每个问题的知识得分
- 根据知识水平调整学习信号，未知时鼓励说‘我不知道’
- 在保持答对率同时显著提升不确定性的判断能力
尽管大语言模型在各类用户提问中表现强大，仍存在幻觉问题，常源于预训练与微调阶段的知识错位。为此，我们通过多轮采样推理，可靠地估算细粒度的实例级知识分数。基于该分数，按模型已有知识水平动态缩放学习信号，并鼓励对超出范围的问题明确回应‘我不知道’。实验表明，该方法使模型在缺乏知识时能准确表达不确定性，同时维持可答问题的高准确率。此外，我们提出不确定性评估指标，证明对已知与未知实例的精准区分可持续提升性能。
原文摘要 · Abstract (English)
While large language models (LLMs) demonstrate strong capabilities across diverse user queries, they still suffer from hallucinations, often arising from knowledge misalignment between pre-training and fine-tuning. To address this misalignment, we reliably estimate a fine-grained, instance-level knowledge score via multi-sampled inference. Using the knowledge score, we scale the learning signal according to the model's existing knowledge, while encouraging explicit "I don't know" responses for out-of-scope queries. Experimental results show that this approach allows the model to explicitly express uncertainty when it lacks knowledge, while maintaining accuracy on questions it can answer. Furthermore, we propose evaluation metrics for uncertainty, showing that accurate discrimination between known and unknown instances consistently improves performance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。