arXiv:2503.19633cs.CL2025-03被引 57

140万条高质量推理数据助力大模型训练,效果超越多个基线模型。

1.4 Million Open-Source Distilled Reasoning Dataset to Empower Large Language Model Training

  • 从开源数据集收集并清洗140万道推理题,保留思维过程
  • 用模型生成答案并通过多方式验证,确保质量
  • 仅用SFT训练即提升性能,适合推理增强研究者使用

AM-DeepSeek-R1-Distilled 是一个包含140万条通用推理任务的大型数据集,涵盖高质且具挑战性的推理问题。这些问题来自多个开源数据源,经过语义去重和精细清洗,避免测试集污染。所有回答均由推理模型(以DeepSeek-R1为主)生成,并通过严格验证:数学题比对参考答案,代码题用测试用例验证,其他任务借助奖励模型评估。仅通过简单监督微调(SFT)训练的AM-Distill-Qwen-32B模型,在AIME2024、MATH-500、GPQA-Diamond和LiveCodeBench四个基准上均优于DeepSeek-R1-Distill-Qwen-32B;AM-Distill-Qwen-72B在所有基准上也超越DeepSeek-R1-Distill-Llama-70B。该数据集已公开于Hugging Face,旨在推动强推理能力大语言模型的发展。

原文摘要 · Abstract (English)

The AM-DeepSeek-R1-Distilled is a large-scale dataset with thinking traces for general reasoning tasks, composed of high-quality and challenging reasoning problems. These problems are collected from a multitude of open-source datasets, subjected to semantic deduplication and meticulous cleaning to eliminate test set contamination. All responses within the dataset are distilled from reasoning models (predominantly DeepSeek-R1) and have undergone rigorous verification procedures. Mathematical problems are validated by checking against reference answers, code problems are verified using test cases, and other tasks are evaluated with the aid of a reward model. The AM-Distill-Qwen-32B model, which was trained through only simple Supervised Fine-Tuning (SFT) using this batch of data, outperformed the DeepSeek-R1-Distill-Qwen-32B model on four benchmarks: AIME2024, MATH-500, GPQA-Diamond, and LiveCodeBench. Additionally, the AM-Distill-Qwen-72B model surpassed the DeepSeek-R1-Distill-Llama-70B model on all benchmarks as well. We are releasing these 1.4 million problems and their corresponding responses to the research community with the objective of fostering the development of powerful reasoning-oriented Large Language Models (LLMs). The dataset was published in \href{https://huggingface.co/datasets/a-m-team/AM-DeepSeek-R1-Distilled-1.4M}{https://huggingface.co/datasets/a-m-team/AM-DeepSeek-R1-Distilled-1.4M}.

推理数据大模型训练模型蒸馏开源数据

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。