用合成数据提升小模型的工具调用能力，效果媲美大模型。
Data Turnstile: A Scalable Open Framework for Function-Calling Data Generation
- 基于API规范生成高质量对话数据，支持多轮交互与错误反馈。
- 小模型用其训练后准确率超基线数倍，接近大模型水平。
- 适合想低成本提升小模型工具使用能力的研究者和开发者。
小语言模型（SLMs）因低延迟、低成本和本地隐私优势，适用于智能体部署，但在工具调用任务中受限于数据稀缺且噪声多。与大模型不同，小模型无法通过规模弥补数据质量缺陷，数据质量成为关键瓶颈。本文提出 Data Turnstile，一个开源框架，可依据用户定义的API规范生成高质量合成训练数据。该框架将多轮工具调用交互分解为受控的分步生成过程，并引入验证与错误反馈机制，实现对API多样性、对话复杂度和输出正确性的精细控制。我们在两个挑战性函数调用基准上验证了其有效性：在BFCL单轮基准上，使用Turnstile数据微调的Qwen3-0.6B模型达到75.9%整体准确率（未启用思维链），优于启用思维链的基线模型（67.4%），接近启用思维链的Qwen3-1.7B（78.4%）和Qwen3-4B（79.9%），尽管其体积分别为后两者1/3和1/7。在τ²-bench多轮智能体基准上，经Turnstile训练的Qwen3-1.7B在电信领域达到31.1% pass^1，较基线6.6%提升4.7倍，并超越19倍更大的Qwen2.5-32B-Instruct（27.4%）。Qwen3-0.6B经训练后达24.6%，较基线3.5%提升7倍，接近53倍大的32B模型。我们公开发布Data Turnstile及涵盖1,000+ API和10万+多轮交互的数据集。
原文摘要 · Abstract (English)
Small language models (SLMs) are attractive for agentic deployment due to low latency, reduced cost, and on-device privacy, yet they struggle with tool-use tasks where training data is scarce and noisy. Unlike larger models, SLMs cannot compensate for low-quality supervision through sheer capacity, making data quality the critical bottleneck. We present Data Turnstile, an open-source framework that takes user-defined API specifications and generates high-quality synthetic training data for function calling. Turnstile decomposes multi-turn tool-use interactions into constrained, stepwise generation with validation and error-feedback loops, providing fine-grained control over API diversity, conversation complexity, and output correctness. We demonstrate effectiveness of domain adaptation with Turnstile data on two challenging function calling benchmarks. On the BFCL single-turn benchmark, a Qwen3-0.6B fine-tuned on Turnstile data without chain-of-thought achieves 75.9% overall accuracy (versus 67.4% for the base model with thinking enabled), closing the gap with thinking-enabled Qwen3-1.7B (78.4%) and Qwen3-4B (79.9%) despite being 3$\times$ and 7$\times$ smaller respectively. On $τ^2$-bench, a multi-turn agentic benchmark, Turnstile-trained Qwen3-1.7B achieves 31.1% pass^1 on the Telecom domain, improving 4.7$\times$ over its 6.6% base and surpassing Qwen2.5-32B-Instruct (27.4%), a model 19$\times$ larger. Turnstile-trained Qwen3-0.6B achieves 24.6%, improving 7$\times$ over its 3.5% base and approaching the 32B model (53$\times$ larger). We release Data Turnstile along with a dataset spanning 1,000+ APIs and 100K+ multi-turn interactions.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。