按推理步骤精准筛选数据，用更少样本达到接近全量训练效果。
GRACE: Gradient-aligned Reasoning Data Curation for Efficient Post-training
- 基于梯度方向和轨迹一致性，逐步评估推理过程价值。
- 仅用5%数据就保持100.2%性能，20%数据达108.8%表现。
- 无需外部标注或奖励模型，适用于多种模型架构。
现有推理数据筛选方法对整个样本进行评分，将每个中间步骤视为等值。实际上，推理链中各步骤贡献不均。本文提出GRACE，将推理轨迹视为一系列优化事件，通过两个互补信号——与目标答案梯度方向的对齐度、与前序推理轨迹的一致性——逐步评分。步骤级分数聚合为样本级得分用于子集选择，仅依赖模型内部优化信号，无需外部奖励模型或步骤标注。为实现可扩展性，GRACE引入表示层梯度代理，仅需一次前向传播即可从标记级上游信号估计步骤级对齐度。在MMathCoT-1M上对Qwen3-VL-2B-Instruct进行后训练，使用20%数据即达到全数据108.8%的性能，仅用5%数据仍保持100.2%，且筛选出的数据集可在不同模型主干间有效迁移。
原文摘要 · Abstract (English)
Existing reasoning data curation pipelines score whole samples, treating every intermediate step as equally valuable. In reality, steps within a trace contribute very unevenly, and selecting reasoning data well requires assessing them individually. We present GRACE, a gradient-aligned curation method that views each reasoning trace as a sequence of optimization events and scores every step by two complementary signals: its alignment with the answer-oriented gradient direction, and its consistency with the preceding reasoning trajectory. Step-level scores are aggregated into a sample-level value for subset selection, using only the model's internal optimization signals and no external reward models or step annotations. To make this scalable, GRACE introduces a representation-level gradient proxy that estimates step-level alignment from token-level upstream signals in a single forward pass. Post-training Qwen3-VL-2B-Instruct on MMathCoT-1M, GRACE reaches 108.8% of the full-data performance with 20% of the data and retains 100.2% with only 5%, with subsets that transfer effectively across model backbones.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。