arXiv:2603.07779cs.CLcs.GL2026-03被引 1

用强化学习筛选难题,让编程模型学得更难更聪明

Scaling Data Difficulty: Improving Coding Models via Reinforcement Learning on Fresh and Challenging Problems

  • 用大模型预测难度,自动挑出有挑战性题目
  • 在新数据上训练300步,性能提升达3倍
  • 适合想提升代码生成能力的研究者

训练下一代代码生成模型需要高质量数据集,但现有数据存在难度不均、格式不一致和质量差的问题。本文通过系统化数据处理与难度扩展,提出四阶段数据处理框架:收集、处理、过滤与验证,其中引入基于大模型的预测-校准-选择机制,结合五维加权难度指标,保留高难度问题并剔除简单题。最终构建的 MicroCoder 数据集包含数万条来自多个平台的真实竞赛题,强调时效性与挑战性。在严格未见的 LiveCodeBench 上评估显示,MicroCoder 在300个训练步骤内相比同规模基线数据集实现3倍性能提升,且在GRPO及其变体算法下均保持优势。对中等和困难题目表现显著优化,不同模型规模下整体性能最高提升17.2%,验证了难度感知的数据清洗能有效提升模型在复杂任务上的表现,为代码生成数据集构建提供多方面启示。

原文摘要 · Abstract (English)

Training next-generation code generation models requires high-quality datasets, yet existing datasets face difficulty imbalance, format inconsistency, and data quality problems. We address these challenges through systematic data processing and difficulty scaling. We introduce a four-stage Data Processing Framework encompassing collection, processing, filtering, and verification, incorporating Automatic Difficulty Filtering via an LLM-based predict-calibrate-select framework that leverages multi-dimensional difficulty metrics across five weighted dimensions to retain challenging problems while removing simplistic ones. The resulting MicroCoder dataset comprises tens of thousands of curated real competitive programming problems from diverse platforms, emphasizing recency and difficulty. Evaluations on strictly unseen LiveCodeBench demonstrate that MicroCoder achieves 3x larger performance gains within 300 training steps compared to widely-used baseline datasets of comparable size, with consistent advantages under both GRPO and its variant training algorithms. The MicroCoder dataset delivers obvious improvements on medium and hard problems across different model sizes, achieving up to 17.2% relative gains in overall performance where model capabilities are most stretched. These results validate that difficulty-aware data curation improves model performance on challenging tasks, providing multiple insights for dataset creation in code generation.

代码生成强化学习数据增强难度筛选

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。