arXiv:2605.31126cs.CLcs.AI2026-05

自生成数据能提升模型能力,但仅当数据与模型匹配时有效。

Not All Synthetic Data Is Yours to Learn From

  • 无提示自训练中,数据与模型需兼容才能生效
  • 自生成数据效果最佳,跨家族迁移明显更差
  • 能力提升与记忆保留可分离,无需刻意遗忘

语言模型能否仅通过自身生成的纯文本(无提示、无教师、无验证器、无奖励模型)进行改进?可以,但前提是合成语料与学生模型兼容——这是一种源模型与学生模型之间的关系属性,而非数据本身的固有特性。我们提出潜在能力重显假说:弱自训练可放大预训练模型已有的能力,但仅在兼容条件下成立。在无提示的无条件自训练最小设定下,基础语言模型仅用起始符(BOS)生成的文本进行微调,无任务指定或外部监督。研究发现:第一,合成数据效用具有相对性而非内在性——自生成数据最有效,同源迁移优于更强但训练方式不同的来源,跨家族迁移显著更弱;第二,常见内在代理指标失效:基准语义相似度与学生模型下的平均词元似然均无法预测哪些语料有助于提升性能;第三,该范式产生意外副产品:在受控的Pythia实验中,能力与逐字记忆实现解耦——基准性能保持或提升,而未见样本的精确匹配提取率下降超95%,且无需遗忘集、隐私目标或针对性遗忘机制。这些结果表明,无提示自训练通过放大学生模型已有知识起作用,而非从数据中引入结构。同时揭示了一种无需显式遗忘目标即可分离能力与记忆的范式。

原文摘要 · Abstract (English)

Can a language model improve from plain text sampled from itself, with no prompts, no teacher, no verifier, and no reward model? Yes, but only when the synthetic corpus is compatible with the student, a relational property of the source-student pair rather than an intrinsic property of the data. We call this the latent capability resurfacing hypothesis: weak self-training can amplify capabilities already present in the pretrained model, but only under this compatibility condition. We study this in the minimal setting of prompt-free unconditional self-training, where base language models are fine-tuned on text generated from the BOS token alone, with no task specification or external supervision. We report three findings. First, synthetic utility is relational rather than intrinsic: self-generated data is the most effective source, same-lineage transfer outperforms stronger but differently trained sources, and cross-family transfer is substantially weaker. Second, common intrinsic proxies fail: neither benchmark-level semantic similarity nor average per-token likelihood under the student predicts which corpora help. Third, this regime produces a surprising byproduct. In controlled Pythia experiments, capability and verbatim memorization decouple: benchmark utility is preserved or improved while held-out exact-match extraction drops by over 95 percent, with no forget set, privacy objective, or targeted unlearning. Together, these results suggest that prompt-free self-training works by amplifying what the student already knows, not by importing structure from the data. They also reveal a regime in which capability and verbatim memorization can be separated without any explicit unlearning objective.

自训练模型能力记忆解耦合成数据

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。