构建超30000条多约束指令数据集,突破大模型复杂指令理解瓶颈。
RECAST: Expanding the Boundaries of LLMs' Complex Instruction Following with Multi-Constraint Data
- 从真实对话中提取约束,生成含19类约束的合成数据
- 模型在30000条数据上微调后,复杂指令遵循能力显著提升
- 支持规则与大模型双重验证,可直接用于强化学习优化
大型语言模型(LLMs)正被期望处理越来越复杂的任务,但当显式要求超过10个时,模型往往难以准确执行,限制了其在真实场景中的应用。现有数据集每例约束数未超过10个。为此,我们提出RECAST框架,高效生成包含远超以往基准数量约束的数据集,以挑战并扩展模型对复杂指令的理解边界。约束源自真实提示-响应对,确保实用性。基于该框架,我们构建了包含3万条实例、覆盖19类约束的RECAST-30K数据集。实验表明,经此数据集微调的模型在复杂指令遵循能力上大幅提升,且通用能力未下降。此外,RECAST支持通过规则验证器(定量约束)和大模型验证器(定性约束)实现自动验证,其可验证性使得可设计奖励函数用于强化学习,进一步提升模型在复杂任务上的表现。
原文摘要 · Abstract (English)
Large language models (LLMs) are increasingly expected to tackle complex tasks, driven by their expanding applications and users' growing proficiency in crafting sophisticated prompts. However, as the number of explicitly stated requirements increases (particularly more than 10 constraints), LLMs often struggle to accurately follow such complex instructions, which limits their applicability in complex real-world scenarios. To the best of our knowledge, existing datasets do not exceed 10 constraints per instance. To address this challenge, we propose RECAST, an efficient and scalable framework for synthesizing datasets where each example incorporates far more constraints than those in existing benchmarks, aiming to challenge and extend the boundaries of models' ability to follow complex instructions. These constraints are extracted from real-world prompt-response pairs to ensure practical relevance. Using this framework, we construct RECAST-30K, a large-scale, high-quality dataset comprising 30k instances spanning 19 constraint types. Experimental results demonstrate that models finetuned on RECAST-30K substantially improve in following complex instructions while maintaining their general capabilities without degradation. Moreover, RECAST enables automatic verification of constraint satisfaction via rule-based validators for quantitative constraints and LLM-based validators for qualitative ones; the verifiability provided by RECAST enables the design of reward functions for reinforcement learning, which further boosts model performance on complex and challenging tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。