让大模型学会自我纠错，训练时故意引入错误与修正对。
Internalized Self-Correction for Large Language Models
- 训练时注入错误及对应修正，形成正负样本对
- 提升指令遵循能力，有效减少幻觉和错误句子
- 无需额外推理步骤，适合部署在实际应用中
本文提出一种名为内部自纠错（InSeC）的大语言模型方法。尽管已有多种推理时自反思的方法，但InSeC结合了负采样、训练时自反思与推理时修正的思想。该方法在训练过程中主动引入错误及其正确版本，使学习过程成为包含正负样本的真实监督学习任务。此机制可有效提升模型对指令的理解能力，并减少生成中的幻觉或语法错误。该方法适用于增强模型生成质量，且不依赖额外推理开销。
原文摘要 · Abstract (English)
In this article, we introduce 'Internalized Self-Correction' (InSeC) for large language models (LLMs). While many approaches exist for self-reflection at inference time, we propose a novel method that combines ideas from negative sampling, self-reflection during training, and inference time. InSeC allows LLMs to correct themselves by introducing mistakes and their corresponding corrections during training, thereby converting the learning process into a true supervised learning task with both positive and negative examples. This approach can be extended to improve instruction following and correct hallucinations or incorrect sentences generated by LLMs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。