用虚构数据+分阶段学习，让大模型更少胡说。
Fictitious Synthetic Data Can Improve LLM Factuality via Prerequisite Learning
- 先学必要知识再练任务技能，避免知识冲突
- 在问答和长文本生成中事实准确率显著提升
- 适合需要高可信度输出的场景，如医疗、法律
近期研究指出，大模型幻觉的一个加剧因素是预训练与微调之间的知识不一致：不熟悉的微调数据会误导模型生成看似合理却错误的内容。本文提出一种新微调策略Prereq-Tune，通过分离技能与知识的学习，使模型仅学习任务技能而不受知识不一致影响。该方法引入额外的前置知识学习阶段，为监督微调（SFT）准备必要知识，使后续SFT专注任务技能。同时，Prereq-Tune可结合虚构合成数据，增强模型输出与内部知识的一致性。实验表明，该方法在短文本问答和长文本生成任务中均优于现有基线，显著提升模型事实性，并为知识可控生成开辟新路径。代码已开源。
原文摘要 · Abstract (English)
Recent studies have identified one aggravating factor of LLM hallucinations as the knowledge inconsistency between pre-training and fine-tuning, where unfamiliar fine-tuning data mislead the LLM to fabricate plausible but wrong outputs. In this paper, we propose a novel fine-tuning strategy called Prereq-Tune to address this knowledge inconsistency and reduce hallucinations. Fundamentally, Prereq-Tune disentangles the learning of skills and knowledge, so the model learns only the task skills without being impacted by the knowledge inconsistency. To achieve this, Prereq-Tune introduces an additional prerequisite learning stage to learn the necessary knowledge for SFT, allowing subsequent SFT to focus only on task skills. Prereq-Tune can also be combined with fictitious synthetic data to enhance the grounding of LLM outputs to their internal knowledge. Experiments show that Prereq-Tune outperforms existing baselines in improving LLM's factuality across short QA and long-form generation tasks. It also opens new possibilities for knowledge-controlled generation in LLMs. Our code is available at https://github.com/UCSB-NLP-Chang/Prereq_tune.git.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。