用真实数据源生成高质量合成数据，提升大模型推理能力。
Source2Synth: Synthetic Data Generation and Curation Grounded in Real Data Sources
- 基于真实数据源生成带推理步骤的合成数据，确保语义合理性。
- 通过答案可回答性筛选，使TQA和MHQA任务性能分别提升25.51%和22.57%。
- 适用于需要复杂推理或工具调用的任务，适合想提升模型泛化能力的研究者。
合成数据生成近年来成为增强大语言模型能力的有前景方法，无需昂贵的人工标注。然而，现有方法常生成质量低或生硬的数据。本文提出Source2Synth，一种可扩展的、基于真实数据源的合成数据生成与筛选方法。该方法输入自定义数据源，生成带有中间推理步骤的合成数据，并通过答案可回答性机制剔除低质样本。我们在两个任务上验证其通用性：多跳问答（MHQA）利用文档测试复杂推理能力，表格问答（TQA）利用表格测试工具使用能力。实验表明，相较于微调基线，该方法在WikiSQL上的TQA任务性能提升25.51%，在HotpotQA上的MHQA任务性能提升22.57%。
原文摘要 · Abstract (English)
Synthetic data generation has recently emerged as a promising approach for enhancing the capabilities of large language models (LLMs) without the need for expensive human annotations. However, existing methods often generate data that can be low quality or contrived. In this paper, we introduce Source2Synth, a scalable approach for synthetic data generation and curation that is grounded in real-world data sources. Source2Synth takes as input a custom data source and produces synthetic data examples with intermediate reasoning steps. Our method improves the dataset quality by discarding low-quality generations based on their answerability. We demonstrate the generality of this approach by applying it to two tasks that leverage two different types of data: multi-hop question answering (MHQA), where we test complex reasoning abilities leveraging documents, and tabular question answering (TQA), where we test tool usage leveraging tables. Our method improves performance by 25.51% for TQA on WikiSQL and 22.57% for MHQA on HotpotQA compared to the fine-tuned baselines.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。