让大模型自己生成并优化训练数据,持续提升性能。
Language Models as Continuous Self-Evolving Data Engineers
- 大模型自主生成、清洗、标注数据,形成闭环自进化。
- 在Qwen2-7B上平均得分提升3.64,在Instruct版提升1.75。
- 无需人工干预,适合追求高效迭代的AI研发团队。
大型语言模型在各类任务中表现卓越,但其进一步发展受限于高质量训练数据的缺乏。传统训练方式过度依赖专家标注数据,制约了模型性能上限。为此,我们提出新型范式LANCE(LANguage models as Continuous self-Evolving data engineers),使大模型能自主生成、清洗、评审和标注带偏好信息的数据,实现自我训练。通过在Qwen2系列模型上进行迭代微调,验证了该方法在多任务上的有效性,显著提升模型性能。在多个基准测试中,Qwen2-7B平均得分提升3.64,Qwen2-7B-Instruct提升1.75。该自主数据构建范式不仅降低对人工专家或外部模型的依赖,还确保数据符合人类偏好,为超越人类能力的未来智能系统发展铺路。代码已开源:https://github.com/Control-derek/LANCE。
原文摘要 · Abstract (English)
Large Language Models (LLMs) have demonstrated remarkable capabilities on various tasks, while the further evolvement is limited to the lack of high-quality training data. In addition, traditional training approaches rely too much on expert-labeled data, setting a ceiling on the performance of LLMs. To address this issue, we propose a novel paradigm named LANCE (LANguage models as Continuous self-Evolving data engineers) that enables LLMs to train themselves by autonomously generating, cleaning, reviewing, and annotating data with preference information. Our approach demonstrates that LLMs can serve as continuous self-evolving data engineers, significantly reducing the time and cost of the post-training data construction. Through iterative fine-tuning on Qwen2 series models, we validate the effectiveness of LANCE across various tasks, showing that it can maintain high-quality data generation and continuously improve model performance. Across multiple benchmark dimensions, LANCE results in an average score enhancement of 3.64 for Qwen2-7B and 1.75 for Qwen2-7B-Instruct. This training paradigm with autonomous data construction not only reduces the reliance on human experts or external models but also ensures that the data aligns with human preferences, paving the way for the development of future superintelligent systems that can exceed human capabilities. Codes are available at: https://github.com/Control-derek/LANCE.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。