LeetCodeDataset提供时序划分的代码生成评测数据集，支持无污染评估与高效微调。
LeetCodeDataset: A Temporal Dataset for Robust Evaluation and Efficient Training of Code LLMs
- 按时间划分数据，实现训练与测试集无污染的严谨评估
- 仅用2.6K模型生成解题方案即可达到110K样本微调效果
- 适合需要强推理能力的代码大模型研究与训练
我们提出LeetCodeDataset，一个高质量的代码生成模型评测与训练基准，解决当前LLM研究中缺乏聚焦推理能力的编码评测标准以及自包含训练环境的问题。通过整理包含丰富元数据、广泛覆盖范围且每道题拥有100多个测试用例的LeetCode Python题目，并采用时间划分（2024年7月前后）设计，该数据集可实现无污染评估和高效的监督微调（SFT）。实验表明，具备推理能力的模型显著优于非推理模型；仅使用2.6K个模型生成的解题方案进行SFT，其性能即接近使用110K样本的微调结果。数据集与评估框架已开源于Hugging Face与Github。
原文摘要 · Abstract (English)
We introduce LeetCodeDataset, a high-quality benchmark for evaluating and training code-generation models, addressing two key challenges in LLM research: the lack of reasoning-focused coding benchmarks and self-contained training testbeds. By curating LeetCode Python problems with rich metadata, broad coverage, 100+ test cases per problem, and temporal splits (pre/post July 2024), our dataset enables contamination-free evaluation and efficient supervised fine-tuning (SFT). Experiments show reasoning models significantly outperform non-reasoning counterparts, while SFT with only 2.6K model-generated solutions achieves performance comparable to 110K-sample counterparts. The dataset and evaluation framework are available on Hugging Face and Github.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。