小模型比大模型更擅长生成高质量指令。
Smaller Language Models Are Better Instruction Evolvers
- 用小模型生成指令，效果优于大模型。
- 小模型能产生更复杂多样的指令变体。
- 提出新评估指标，更准确衡量指令质量。
指令微调已被广泛用于释放大语言模型的全部潜力。值得注意的是，复杂多样的指令对有效对齐模型与下游任务至关重要。然而，当前大规模指令构建方法主要依赖强大模型（如GPT-4或参数超过700亿的模型），基于一种经验假设：大模型天生具备更强能力。本研究质疑这一普遍假设，深入探索小语言模型（SLMs）在指令演化中的潜力。在三种指令演化场景下的大量实验表明，小语言模型生成的指令比大模型更有效。进一步分析显示，小模型在指令演化过程中具有更广阔的输出空间，从而生成更复杂、更多样化的指令变体。我们还发现，现有评估指标未能关注指令的实际影响。因此，提出新的评估方法IC-IFD，将指令复杂度引入原始IFD分数，以更准确评估指令数据的有效性。代码已开源。
原文摘要 · Abstract (English)
Instruction tuning has been widely used to unleash the complete potential of large language models. Notably, complex and diverse instructions are of significant importance as they can effectively align models with various downstream tasks. However, current approaches to constructing large-scale instructions predominantly favour powerful models such as GPT-4 or those with over 70 billion parameters, under the empirical presumption that such larger language models (LLMs) inherently possess enhanced capabilities. In this study, we question this prevalent assumption and conduct an in-depth exploration into the potential of smaller language models (SLMs) in the context of instruction evolution. Extensive experiments across three scenarios of instruction evolution reveal that smaller language models (SLMs) can synthesize more effective instructions than LLMs. Further analysis demonstrates that SLMs possess a broader output space during instruction evolution, resulting in more complex and diverse variants. We also observe that the existing metrics fail to focus on the impact of the instructions. Thus, we propose Instruction Complex-Aware IFD (IC-IFD), which introduces instruction complexity in the original IFD score to evaluate the effectiveness of instruction data more accurately. Our source code is available at: \href{https://github.com/HypherX/Evolution-Analysis}{https://github.com/HypherX/Evolution-Analysis}
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。