让大模型从连续任务中积累经验,提升生成数据能力。
Make LLM Learn to Synthesize from Streaming Experiences through Feedback

- 通过持续学习历史任务经验,动态优化生成策略。
- 跨任务迁移效果显著,后期任务生成质量提升明显。
- 适合需要长期优化数据生成的自动化系统使用。
大语言模型广泛用于合成数据生成,大幅降低标注成本。然而,现有研究多将合成视为孤立任务,忽视了模型能否通过累积过往经验、迁移到未来任务的核心问题。本文提出StreamSynth新范式:合成任务按序到来,历史经验为后续合成提供有效信号。为此,我们设计SynLearner框架,使合成模型能从任务流中获取可复用的经验。与独立生成不同,SynLearner鼓励模型探索多样生成模式,根据反馈学习,并在任务演进中平衡样本质量与集合级多样性。在多个基准上的实验表明,SynLearner能有效利用早期任务经验,显著提升后期任务的合成性能,展现出稳定的跨任务迁移能力。这些结果验证了StreamSynth的可行性,表明合成数据生成可作为经验驱动过程,受益于任务流的持续积累。
原文摘要 · Abstract (English)
Large language models (LLMs) have been widely adopted for synthetic data generation, significantly reducing annotation costs. However, most existing studies treat synthesis as a set of isolated tasks and overlook a more fundamental question: whether a model can learn to synthesize by accumulating experience from past tasks and transferring it to future ones. In this work, we introduce StreamSynth, a new setting in which synthesis tasks arrive sequentially and experience from historical tasks provides informative signals for future synthesis. To address this setting, we propose SynLearner, a general framework that enables synthesis models to acquire reusable synthesis experience over a task stream. Instead of generating data independently for each task, SynLearner encourages the model to explore diverse synthesis patterns, learn from feedback, and balance sample quality with set-level diversity as tasks evolve. Extensive experiments across multiple benchmarks show that SynLearner effectively leverages experience from earlier tasks to improve synthesis performance on later ones, exhibiting consistent cross-task transferability. These findings provide evidence for the feasibility of StreamSynth and highlight synthetic data generation as an experience-driven process that can benefit from task streams.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。