arXiv:2605.13149cs.CLcs.AI2026-05被引 1

用主动学习方法生成更高质量的合成数据,让模型自己优化训练数据。

AcquisitionSynthesis: Targeted Data Generation using Acquisition Functions

论文配图:AcquisitionSynthesis: Targeted Data Generation using Acquisition Functions
图 1 · 摘自论文原文
  • 用采集函数作奖励信号,指导语言模型生成更有价值的合成数据。
  • 生成数据使下游模型在数学、医疗问答等任务上提升2-7%且更抗遗忘。
  • 适合需要高质量自生成数据的研究者,尤其关注模型鲁棒性与自我迭代。

数据质量仍是构建高性能模型的关键瓶颈。现有方法多依赖拒绝采样或大模型挖掘弱点来生成数据,但缺乏对生成样本影响下游模型的量化评估。主动学习中的采集函数可衡量数据的信息量与影响力,提供可解释的模型中心信号。受此启发,本文提出AcquisitionSynthesis:将采集函数作为奖励模型,训练语言模型生成更高质量的合成数据。我们在数学、医疗问答和编码等经典可验证任务上进行实验,结果表明:(1) 使用AcquisitionSynthesis生成数据训练的学生模型在分布内任务上表现优异(提升2-7%),且对灾难性遗忘更具鲁棒性;(2) 该方法可为其他模型生成数据,适用于低资源到高资源的训练范式。通过利用采集函数奖励,我们展示了一条模型感知的、超越静态数据集的自我改进路径。

原文摘要 · Abstract (English)

Data quality remains a critical bottleneck in developing capable, competitive models. Researchers have explored many ways to generate top quality samples. Some works rely on rejection sampling: generating lots of synthetic samples and filtering out low-quality samples. Other works rely on larger or closed-source models to extract model weaknesses, necessary skills, or a curriculum off of which to base data generation. These works have one common limitation: there is no quantitative approach to measure the impact of the generated samples on the downstream learner. Active learning literature provides exactly this, in the form of acquisition functions. Acquisition functions measure the informativeness and/or influence of data, providing interpretable, model-centric signals. Inspired by this, we propose AcquisitionSynthesis: using acquisition functions as reward models to train language models to generate higher-quality synthetic data. We conduct experiments on classic verifiable tasks of math, medical question-answering, and coding. Our experimental results indicate that (1) student models trained with AcquisitionSynthesis data achieve good performance on in-distribution tasks (2-7% gain) and is more robust to catastrophic forgetting, and (2) AcquisitionSynthesis models can generate data for other models and for low-to-high resource training paradigms. By leveraging acquisition rewards, we seek to demonstrate a principled path toward model-aware self-improvement that surpasses static datasets.

数据生成主动学习语言模型自提升

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。