构建5000万条合成对话数据集，验证其在大模型微调中的高效性
WILDCHAT-50M: A Deep Dive Into the Role of Synthetic Data in Post-Training
- 构建包含50多个开源模型的5000万条合成对话数据集
- 用40%样本量实现优于Tulu-3 SFT混合数据的性能
- 为大模型后训练提供可复现的公开数据与代码
大语言模型后训练（如DPO、蒸馏）能优化行为并解锁新能力，但相关开放研究仍处于初期。主要瓶颈在于难以大规模对比合成数据生成模型与大模型评判器。为此，我们推出目前最大的公开聊天数据集WILDCHAT-50M，将现有WildChat数据扩展至涵盖50多个不同规模的开源模型（参数量从0.5B到104B），生成超过5000万条对话响应。我们开展全面对比分析，并基于此构建了名为RE-WILD的公开SFT混合数据集，仅使用40%的样本量即超越了Allen AI发布的最新版Tulu-3 SFT混合数据集的表现。数据集、样本及代码已开源。
原文摘要 · Abstract (English)
Language model (LLM) post-training, from DPO to distillation, can refine behaviors and unlock new skills, but the open science supporting these post-training techniques is still in its infancy. One limiting factor has been the difficulty of conducting large-scale comparative analyses of synthetic data generating models and LLM judges. To close this gap, we introduce WILDCHAT-50M, the largest public chat dataset to date. We extend the existing WildChat dataset to include responses not only from GPT, but from over 50 different open-weight models, ranging in size from 0.5B to 104B parameters. We conduct an extensive comparative analysis and demonstrate the potential of this dataset by creating RE-WILD, our own public SFT mix, which outperforms the recent Tulu-3 SFT mixture from Allen AI with only 40% as many samples. Our dataset, samples and code are available at https://github.com/penfever/wildchat-50m.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。