让小模型精准生成指定格式内容，靠自动验证数据提升能力
Verifiable Format Control for Large Language Model Generations
- 构建可自动验证的格式数据集VFF，每条数据用Python函数即可校验
- 用该数据集持续训练7B小模型，格式遵循准确率显著提升
- 适合需要可靠结构化输出的应用场景，如自动化脚本生成
近期大语言模型在通用指令遵循方面表现良好，但参数量约70亿的小模型仍难以精确遵循细粒度格式（如JSON），严重制约其应用发展。现有方法多聚焦通用指令遵循评估，忽视小模型的格式遵循能力提升，且常依赖GPT-4等高级模型进行评测，存在内在偏差且成本高昂。本文首次构建了全可验证的格式遵循数据集VFF，与以往依赖外部LLM验证不同，VFF中每条样本均可通过Python函数快速校验。进一步，利用该可验证特性，我们设计方法合成海量训练数据，用于逐步训练小模型以增强其格式遵循能力。实验表明，70亿级开源模型普遍存在格式遵循缺陷，而本文方法能有效提升该关键能力。
原文摘要 · Abstract (English)
Recent Large Language Models (LLMs) have demonstrated satisfying general instruction following ability. However, small LLMs with about 7B parameters still struggle fine-grained format following (e.g., JSON format), which seriously hinder the advancements of their applications. Most existing methods focus on benchmarking general instruction following while overlook how to improve the specific format following ability for small LLMs. Besides, these methods often rely on evaluations based on advanced LLMs (e.g., GPT-4), which can introduce the intrinsic bias of LLMs and be costly due to the API calls. In this paper, we first curate a fully verifiable format following dataset VFF. In contrast to existing works often adopting external LLMs for instruction-following validations, every sample of VFF can be easily validated with a Python function. Further, we propose to leverage this verifiable feature to synthesize massive data for progressively training small LLMs, in order to improve their format following abilities. Experimental results highlight the prevalent limitations in the format following capabilities of 7B level open-source LLMs and demonstrate the effectiveness of our method in enhancing this essential ability.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。