用分级难度数据训练大模型，显著提升数学推理能力。
DeepDistill: Enhancing LLM Reasoning Capabilities via Large-Scale Difficulty-Graded Data Training
- 构建334万条不同难度的推理数据，通过多轮模型生成4000万条响应。
- 精选高质量数据后，模型在AIME2024上通过率达79.2%，接近顶尖水平。
- 公开全部数据与方法，推动开源长推理模型发展。
尽管大语言模型在复杂推理基准上表现优异，但学界对基础模型训练过程和数据质量仍缺乏深入理解。为此，我们构建了一个大规模、难度分级的推理数据集，包含约334万条不同难度的独特查询及约4000万条由多模型多轮生成的提炼响应。基于通过率与变异系数（CV）精确筛选高价值训练数据，以增强推理能力。值得注意的是，我们观察到训练模式转变：基于基础模型的推理训练需更高学习率才能有效。利用此精心筛选的数据，我们显著提升了基础模型的推理能力，在AIME2024数学推理基准上达到79.2%的通过率，超越多数现有蒸馏模型，并接近当前最先进水平。我们详细描述了数据处理、难度评估与训练方法，并公开所有数据与代码，以促进开源长推理大模型的快速进展。数据集获取地址：https://huggingface.co/datasets/a-m-team/AM-DeepSeek-Distilled-40M
原文摘要 · Abstract (English)
Although large language models (LLMs) have recently achieved remarkable performance on various complex reasoning benchmarks, the academic community still lacks an in-depth understanding of base model training processes and data quality. To address this, we construct a large-scale, difficulty-graded reasoning dataset containing approximately 3.34 million unique queries of varying difficulty levels and about 40 million distilled responses generated by multiple models over several passes. Leveraging pass rate and Coefficient of Variation (CV), we precisely select the most valuable training data to enhance reasoning capability. Notably, we observe a training pattern shift, indicating that reasoning-focused training based on base models requires higher learning rates for effective training. Using this carefully selected data, we significantly improve the reasoning capabilities of the base model, achieving a pass rate of 79.2\% on the AIME2024 mathematical reasoning benchmark. This result surpasses most current distilled models and closely approaches state-of-the-art performance. We provide detailed descriptions of our data processing, difficulty assessment, and training methodology, and have publicly released all datasets and methods to promote rapid progress in open-source long-reasoning LLMs. The dataset is available at: \href{https://huggingface.co/datasets/a-m-team/AM-DeepSeek-Distilled-40M}{https://huggingface.co/datasets/a-m-team/AM-DeepSeek-Distilled-40M}
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。