用大模型生成文本和代码合成数据,解决真实数据不足问题。
Synthetic Data Generation Using Large Language Models: Advances in Text and Code
- 通过提示工程、检索增强等方法生成任务相关的合成数据。
- 提升低资源场景下的分类与问答性能,支持代码修复与翻译。
- 适合数据稀缺或敏感场景的研究者与开发者使用。
本文综述大语言模型(LLMs)在自然语言与代码领域生成合成训练数据的最新进展。通过生成人工但任务相关的样本,这些模型可显著扩充或替代真实数据集,尤其在标注数据稀缺、昂贵或敏感的场景中。文章重点介绍提示驱动生成、检索增强管道及迭代自精炼等关键技术,探讨其在低资源任务(如分类、问答)和代码应用(如指令微调、代码翻译、漏洞修复)中的价值,结合自动化验证确保功能正确性。尽管具有成本低、覆盖广、多样性可控等优势,仍面临事实错误、风格不真实及偏见放大等问题。应对策略包括输出过滤加权、代码域中的执行反馈强化学习。最后,提出自动提示工程、跨模态数据合成与鲁棒评估框架等开放方向,强调合成数据在加速AI发展中的重要性,并需加强伦理与质量保障。
原文摘要 · Abstract (English)
This survey reviews how large language models (LLMs) are transforming synthetic training data generation in both natural language and code domains. By producing artificial but task-relevant examples, these models can significantly augment or even substitute for real-world datasets, particularly in scenarios where labeled data is scarce, expensive, or sensitive. This paper surveys recent advances in leveraging LLMs to create synthetic text and code, highlighting key techniques such as prompt-based generation, retrieval-augmented pipelines, and iterative self-refinement. We examine how these methods can enrich low-resource tasks (e.g., classification, question answering) and facilitate code-centric applications (e.g., instruction tuning, code translation, bug repair) through automated verification of functional correctness. Alongside potential benefits - cost-effectiveness, broad coverage, and controllable diversity - we discuss the accompanying challenges, including factual inaccuracies in generated text, insufficient stylistic or distributional realism, and risks of bias amplification. Proposed mitigation strategies range from filtering and weighting synthetic outputs to reinforcement learning with execution feedback in code domains. We conclude by outlining open research directions, such as automated prompt engineering, cross-modal data synthesis, and robust evaluation frameworks, underscoring the growing importance of LLM-generated synthetic data in accelerating AI development while emphasizing ethical and quality safeguards.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。