用对比解码生成合成数据，提升低资源语言模型性能
Contrastive Decoding for Synthetic Data Generation in Low-Resource Language Modeling
- 通过对比好差模型的输出差异生成合成数据
- 混合真实与合成数据后，语言建模和下游任务均提升
- 对比解码适合需要推理的任务，传统采样适合表层语言任务
大语言模型（LLMs）依赖海量文本数据训练，而数据增长可能已达瓶颈。本文探索利用LLM自身生成合成数据的可行性。在控制实验中，使用1亿词原始语料训练两个模型：一个表现较好，一个较差。通过对比解码放大优质模型的输出信号，生成合成语料，并与原始数据混合训练。结果表明，混合数据训练显著提升语言建模效果及多种下游任务性能。尤其发现，对比解码生成的数据更有利于需要深度推理的任务，而传统采样生成的数据对表层语言能力相关任务更有帮助。
原文摘要 · Abstract (English)
Large language models (LLMs) are trained on huge amounts of textual data, and concerns have been raised that the limits of such data may soon be reached. A potential solution is to train on synthetic data sampled from LLMs. In this work, we build on this idea and investigate the benefits of contrastive decoding for generating synthetic corpora. In a controlled setting, we experiment with sampling corpora using the relative difference between a good and bad model trained on the same original corpus of 100 million words. By amplifying the signal from a model that has better performance, we create a synthetic corpus and mix it with the original training data. Our findings show that training on a mixture of synthesized and real data improves performance on the language modeling objective and a range of downstream tasks. In particular, we see that training with a mix of synthetic data from contrastive decoding benefits tasks that require more reasoning skills, while synthetic data from traditional sampling helps more on tasks dependent on surface level linguistic capabilities.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。