动态调整训练数据，让模型学得更准更省
Dynamic Sampling that Adapts: Self-Aware Iterative Data Persistent Optimization for Mathematical Reasoning
- 根据模型当前能力实时选题，不再死板用固定标准
- 在8个数学竞赛数据集上提升近6分，用更少数据达到顶尖效果
- 适合想高效训练数学推理模型的研究者和工程师
在数学推理任务中，数据选择通常依赖静态外部指标，无法随模型能力变化而调整，导致监督微调和强化学习效率受限。为此，我们提出SAI-DPO（Self-Aware Iterative Data Persistent Optimization）——一种动态采样框架，使训练数据与模型内在能力相匹配。该框架引入两个新指标：知识语义对齐（用于识别领域薄弱点），以及基于通过率和推理路径特征的自知难度（衡量题目复杂度相对于模型当前状态）。通过实时反馈不断重校数据分布，SAI-DPO确保训练样本始终贴合模型当前能力。在包含AIME24和AMC23在内的8个基准上的实验表明，SAI-DPO性能优于静态基线近6个百分点，以显著更少的数据实现顶尖效率。
原文摘要 · Abstract (English)
In mathematical reasoning, data selection strategies predominantly rely on static, externally defined metrics, which fail to adapt to the evolving capabilities of models during training. This misalignment limits the efficiency of Supervised Fine-Tuning and Reinforcement Learning. To bridge this gap, we introduce SAI-DPO (Self-Aware Iterative Data Persistent Optimization), a dynamic sampling framework that aligns training data with the model's intrinsic competence. SAI-DPO operationalizes two novel metrics: Knowledge Semantic Alignment for targeting domain weaknesses, and Self-Aware Difficulty, derived from pass rates and reasoning path characteristics, to gauge instance complexity relative to the model's current state. By iteratively recalibrating the data distribution based on real-time feedback, SAI-DPO dynamically aligns training samples with the model's evolving competence, ensuring the data remains strictly relevant to the model's current capability level. Extensive experiments on eight benchmarks (including AIME24 and AMC23) demonstrate that SAI-DPO outperforms static baselines at most nearly 6 points, achieving state-of-the-art efficiency with significantly less data.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。