通过分解复杂数据集，逐步训练模型解决难题。
Learning to Solve Complex Problems via Dataset Decomposition
- 用分步推理的教师模型递归拆解复杂问题为简单子任务。
- 在MATH和AIME数据集上，新方法训练的模型准确率显著提升。
- 适合需要渐进学习的数学与代码生成任务研究者。
课程学习是一种按难度从易到难组织训练数据的策略。本文提出一种逆向课程生成方法，通过递归分解复杂数据集为更简单、可学性强的组件。我们设计了一个具备分步推理能力的教师-学生框架，教师用于生成更简单的例子，使学生模型逐步掌握困难任务。提出一种基于结构复杂度与概念深度的新评分系统，实现对分解后数据的课程构建。在数学数据集（MATH 和 AIME）及代码生成数据集上的实验表明，使用该方法生成的课程训练出的模型，在性能上优于在原始数据集上标准训练的模型。
原文摘要 · Abstract (English)
Curriculum learning is a class of training strategies that organizes the data being exposed to a model by difficulty, gradually from simpler to more complex examples. This research explores a reverse curriculum generation approach that recursively decomposes complex datasets into simpler, more learnable components. We propose a teacher-student framework where the teacher is equipped with the ability to reason step-by-step, which is used to recursively generate easier versions of examples, enabling the student model to progressively master difficult tasks. We propose a novel scoring system to measure data difficulty based on its structural complexity and conceptual depth, allowing curriculum construction over decomposed data. Experiments on math datasets (MATH and AIME) and code generation datasets demonstrate that models trained with curricula generated by our approach exhibit superior performance compared to standard training on original datasets.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。