让微调目标匹配模型已知知识，可有效减少幻觉。
Stick to What You Know: A Study of Knowledge-Aligned Supervised Fine-Tuning
- 用模型自身知识约束微调目标，避免生成未知内容
- 在WildHalu和Biography数据集上显著降低事实性错误
- 适合关注模型可信度与安全性的人参考
监督微调（SFT）通过让基础语言模型模仿目标回复来提升性能，但这些目标可能包含基础模型未充分内化的知识，成为幻觉的来源。本文将其视为问题，并提出‘知识对齐的SFT’：将SFT训练目标限制在基础模型的参数化知识范围内。在统一框架下，比较了基于生成与基于估计的知识对齐方法，并引入两种新方法：Evidence Rewrite利用外部证据验证模型生成；Recall Rewrite仅保留模型能一致回忆的内容。在Qwen 3 4B和OLMo 3 7B上的实验表明，知识对齐的SFT可在减少WildHalu和Biography数据集上事实性幻觉的同时，基本保持通用能力。Recall Rewrite表现最佳，显著提升事实准确性并改善UnknownBench上的拒绝行为。结果证实：超出模型知识范围的SFT目标会引发幻觉。
原文摘要 · Abstract (English)
Supervised fine-tuning (SFT) trains a base language model to imitate target responses, and these targets may require knowledge the base model has not robustly internalized. We study this as a source of hallucinations and frame a group of mitigation methods as \emph{knowledge-aligned SFT}: constraining SFT training targets to the base model's parametric knowledge. Under a unified setup, we compare existing generation-based and estimation-based knowledge-alignment methods and introduce two new variants: Evidence Rewrite, which verifies base-model generations using external evidence, and Recall Rewrite, which retains claims only when they can be consistently recalled by the base model. Experiments with Qwen 3 4B and OLMo 3 7B show that knowledge-aligned SFT can reduce factual hallucinations on WildHalu and Biography while largely preserving general capabilities. Recall Rewrite yields the strongest factuality gains and improves refusal behavior on UnknownBench. It thereby confirms that SFT targets beyond the base model's knowledge drive hallucination behavior.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。