用开源数据让小模型学会复杂指令，性能追平大厂模型。
UltraIF: Advancing Instruction Following from the Wild
- 将用户指令拆解为查询、约束和评估问题，构建可组合的提示生成器。
- 仅用80亿参数模型在5个基准上达到与指令版对齐模型相当的性能。
- 方法可自增强，适合资源有限的研究者提升开源模型能力。
指令跟随使大型语言模型成为得力助手，但如何让模型理解复杂指令仍不明确，开源模型与头部公司训练模型间存在巨大差距。为此，我们提出一种简单且可扩展的方法UltraIF，利用开源数据构建能遵循复杂指令的LLM。UltraIF首先将真实用户提示分解为简单查询、约束及其对应的评估问题；随后训练一个UltraComposer，用于组合关联约束的提示与评估问题。该提示生成器可合成复杂指令并用评估问题筛选回复。实验首次证明，仅使用80亿参数模型作为生成器和评估器，在无基准信息条件下，将LLaMA-3.1-8B-Base对齐至其指令版水平，并在多个其他基准上取得有竞争力的表现。此外，UltraIF还可通过自对齐进一步提升LLaMA-3.1-8B-Instruct性能，拓展应用场景。代码已公开于https://github.com/kkk-an/UltraIF。
原文摘要 · Abstract (English)
Instruction-following made modern large language models (LLMs) helpful assistants. However, the key to taming LLMs on complex instructions remains mysterious, for that there are huge gaps between models trained by open-source community and those trained by leading companies. To bridge the gap, we propose a simple and scalable approach UltraIF for building LLMs that can follow complex instructions with open-source data. UltraIF first decomposes real-world user prompts into simpler queries, constraints, and corresponding evaluation questions for the constraints. Then, we train an UltraComposer to compose constraint-associated prompts with evaluation questions. This prompt composer allows us to synthesize complicated instructions as well as filter responses with evaluation questions. In our experiment, for the first time, we successfully align LLaMA-3.1-8B-Base to catch up with its instruct version on 5 instruction-following benchmarks without any benchmark information, using only 8B model as response generator and evaluator. The aligned model also achieved competitive scores on other benchmarks. Moreover, we also show that UltraIF could further improve LLaMA-3.1-8B-Instruct through self-alignment, motivating broader use cases for the method. Our code is available at https://github.com/kkk-an/UltraIF.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。