arXiv:2504.11919cs.AI2025-04被引 11

根据大模型能力动态调整题目难度,高效生成高质量思维链数据

Rethinking the Generation of High-Quality CoT Data from the Perspective of LLM-Adaptive Question Difficulty Grading

  • 用大模型自身推理能力评估题目难度,构建自适应题库
  • 仅用2000条数据即让小模型超越原版32B模型的数学与代码推理能力
  • 适合需要低成本提升小模型推理能力的研究者和开发者

近期,DeepSeek-R1 (671B)(DeepSeek-AI et al., 2025)在复杂任务中展现出优异的推理能力,并公开了其方法。这为小型大语言模型(LLMs)的推理能力训练提供了潜在的高质量思维链(CoT)数据。为生成适配不同大模型的高质量CoT数据,本文提出一种基于大模型自适应题目难度分级的方法。首先,依据大模型自身的推理能力对题目进行难度评级,构建大模型自适应题库;其次,根据题目难度分布采样,并利用DeepSeek-R1 (671B)生成带正确答案的高质量CoT数据。得益于该自适应难度的CoT数据构建方式,显著降低了数据生成成本,提升了模型监督微调(SFT)效率。最终验证了该方法在复杂数学竞赛与代码生成任务中的有效性与泛化性。值得注意的是,仅使用2000条高质量数学CoT数据,我们的ZMath-32B在数学推理任务上超过DeepSeek-Distill-32B;同样,仅用2000条高质量代码CoT数据,ZCode-32B在代码推理任务上也优于后者。

原文摘要 · Abstract (English)

Recently, DeepSeek-R1 (671B) (DeepSeek-AIet al., 2025) has demonstrated its excellent reasoning ability in complex tasks and has publiclyshared its methodology. This provides potentially high-quality chain-of-thought (CoT) data for stimulating the reasoning abilities of small-sized large language models (LLMs). To generate high-quality CoT data for different LLMs, we seek an efficient method for generating high-quality CoT data with LLM-Adaptive questiondifficulty levels. First, we grade the difficulty of the questions according to the reasoning ability of the LLMs themselves and construct a LLM-Adaptive question database. Second, we sample the problem database based on a distribution of difficulty levels of the questions and then use DeepSeek-R1 (671B) (DeepSeek-AI et al., 2025) to generate the corresponding high-quality CoT data with correct answers. Thanks to the construction of CoT data with LLM-Adaptive difficulty levels, we have significantly reduced the cost of data generation and enhanced the efficiency of model supervised fine-tuning (SFT). Finally, we have validated the effectiveness and generalizability of the proposed method in the fields of complex mathematical competitions and code generation tasks. Notably, with only 2k high-quality mathematical CoT data, our ZMath-32B surpasses DeepSeek-Distill-32B in math reasoning task. Similarly, with only 2k high-quality code CoT data, our ZCode-32B surpasses DeepSeek-Distill-32B in code reasoning tasks.

思维链大模型训练推理增强数据生成

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。