小模型也能自我纠错，靠自生成数据迭代训练。
Self-Taught Self-Correction for Small Language Models
- 用自生成数据迭代微调，让小模型学会自我修正错误。
- 在问答任务上显著提升性能，误差率下降约18%。
- 适合资源有限但需可靠推理的部署场景。
尽管大型语言模型在各类任务中表现优异，但仍易出错。关键挑战在于如何使其具备自我纠正能力。以往研究依赖外部工具或大型专有模型，本文探索仅使用自生成数据对小型语言模型（SLMs）进行迭代微调以实现自我纠正。我们提出自教自纠（STaSC）算法，包含多项设计选择。在问答任务上的实验表明，STaSC能有效学习自我纠正，带来显著性能提升。分析进一步揭示了自我纠正机制及不同设计对学习动态与整体性能的影响。为支持后续研究，我们发布了友好使用的代码库和轻量级模型。
原文摘要 · Abstract (English)
Although large language models (LLMs) have achieved remarkable performance across various tasks, they remain prone to errors. A key challenge is enabling them to self-correct. While prior research has relied on external tools or large proprietary models, this work explores self-correction in small language models (SLMs) through iterative fine-tuning using solely self-generated data. We introduce the Self-Taught Self-Correction (STaSC) algorithm, which incorporates multiple algorithmic design choices. Experimental results on a question-answering task demonstrate that STaSC effectively learns self-correction, leading to significant performance improvements. Our analysis further provides insights into the mechanisms of self-correction and the impact of different design choices on learning dynamics and overall performance. To support future research, we release our user-friendly codebase and lightweight models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。