提升指令模型性能的关键是让指令与回复高度匹配。
MAIN: Mutual Alignment Is Necessary for instruction tuning
- 通过相互约束机制强制指令与回复保持一致
- 在多个大模型上达到当前最佳表现
- 适合需要高质量指令数据的研究者
指令微调已使大语言模型（LLMs）取得显著性能提升，但其成功高度依赖大规模、高质量的指令-回复对。为满足这一需求，已有多种方法用于大规模数据合成，但现有方法常忽视指令与回复之间的对齐问题。我们提出互对齐框架（MAIN），通过相互约束强化指令与回复间的连贯性。实验表明，MAIN在不同模型架构与规模下均具有良好泛化能力，在LLaMA、Mistral和Qwen等模型上于多个基准测试中达到领先水平。该研究强调了指令-回复对齐在实现可泛化、高质量指令微调中的关键作用。所有代码均已开源。
原文摘要 · Abstract (English)
Instruction tuning has empowered large language models (LLMs) to achieve remarkable performance, yet its success heavily depends on the availability of large-scale, high-quality instruction-response pairs. To meet this demand, various methods have been developed to synthesize data at scale. However, current methods for scaling up data generation often overlook a crucial aspect: the alignment between instructions and responses. We hypothesize that the quality of instruction-response pairs is determined not by the individual quality of each component, but by the degree of mutual alignment. To address this, we propose a Mutual Alignment Framework (MAIN) which enforces coherence between instructions and responses through mutual constraints. We demonstrate that MAIN generalizes well across model architectures and sizes, achieving state-of-the-art performance on LLaMA, Mistral, and Qwen models across diverse benchmarks. This work underscores the critical role of instruction-response alignment in enabling generalizable and high-quality instruction tuning for LLMs. All code is available from our repository.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。