用真实代码场景生成复杂多样的编程题目
SCoGen: Scenario-Centric Graph-Based Synthesis of Real-World Code Problems
- 基于真实数据构建领域、技能与代码的关联图谱
- 生成问题在多个基准上优于主流大模型
- 适合需要真实场景训练的代码生成研究者
代码大模型虽已取得显著进展,但其进一步发展受限于真实编程问题的稀缺。为此,我们提出SCoGen框架,通过从Stack Overflow和Kaggle等真实编程数据集中提取领域知识、领域技能与编码技能,构建场景为中心的关联图谱。该图谱整合了应用情境,设计采样策略以控制生成问题的复杂度与多样性,使其贴近实际开发挑战。实验表明,该方法在多个真实世界基准测试中持续优于不同规模与功能的开源大模型,涵盖专用编码模型与通用模型。
原文摘要 · Abstract (English)
Significant advancements have been made in the capabilities of code large language models, leading to their rapid adoption and application across a wide range of domains. However, their further advancements are often constrained by the scarcity of real-world coding problems. To bridge this gap, we propose a novel framework for synthesizing code problems that emulate authentic real-world scenarios. This framework systematically integrates domain knowledge, domain skills, and coding skills, all of which are meticulously extracted from real-world programming-related datasets, including Stack Overflow and Kaggle. The extracted elements serve as the foundational building blocks for constructing code problems. To align the generated problems with practical applications, application scenarios are also mined from the aforementioned datasets. These scenarios are then utilized to construct a scenario-centric graph that interconnects domain knowledge, domain skills, and coding skills. Based on this structured representation, a sampling strategy on the graph is designed, which effectively controls the generation of a code problem with complexity and diversity, reflects real-world challenges. Experimental results demonstrate that the proposed method consistently achieves superior performance over state-of-the-art open-source large language models of varying sizes and functionalities, including both coders and general-purpose models, across a diverse set of real-world benchmarks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。