arXiv:2607.18642cs.LGcs.SE2026-07

生成可控、无污染的多语言代码数据集,确保每段代码正确且可评估。

Spaghetti Architect: A Contamination-Resistant, By-Construction-Labelled, Multi-Language Code Dataset Generator

  • 通过反优化编译器将统一中间表示转为冗余代码,控制复杂度与混乱度。
  • 在四个模型上测试,难度因子使最强模型算术聚合准确率降为0。
  • 生成数据可复现,适合评估代码理解与重构能力,避免训练数据污染。

现有挖掘的代码语料缺乏控制:片段语义、表面混乱度和难度均来自原始数据,无最优参考标准,且公开样本可能已存在于模型训练集中。本文提出Spaghetti Architect,一种可生成受控代码数据集的工具。该工具通过反优化编译器将清洁、语言无关的JSON中间表示(IR)转换为五种语言(Python、JavaScript、Go、Java、C++)中冗余且完全扁平化的程序;每个程序经编译、运行并由参考基准验证,确保其正确性。清洁的IR作为已知最优参考,混乱度由严格嵌套的反模式配置调节,每条实例在两个正交维度上标注难度:内在(问题规模)与附带(表达形式固定语义)。通过私有保留种子生成新变体以抵抗污染。实验表明,生成数据质量顺序符合复杂度与可读性指标;在四模型开放排行榜上,精确匹配率随规模上升,但内在难度因子使最强模型的算术聚合准确率归零。开发集得分与重新生成的保留集相差不超过|Δ|≤0.012(理解)和≤0.011(重构);相同程序上,重构等价性(0.73→0.99)与规模无关,而输出预测性能崩溃;消融分析显示自注释对最弱模型提升一个数量级(-0.173对比-0.017),注释版本可解决三组相邻对中的一个,未注释版本则全解。项目开源(MIT)、无依赖、永久存档于持久DOI。

原文摘要 · Abstract (English)

Mined code corpora are abundant but uncontrolled: a snippet's semantics, surface "messiness," and difficulty are whatever the wild contained; there is no known-optimal reference to grade against; and any public sample may already sit in a model's training set. We present Spaghetti Architect, a tool that mints code datasets with the control such corpora lack. An anti-optimization transpiler maps a clean, language-agnostic JSON intermediate representation to deliberately redundant, fully-flattened programs in five languages (Python, JavaScript, Go, Java, C++); every program is compiled, run, and checked against a reference oracle, so each instance is correct by construction. The clean IR is a known-optimal reference, messiness is dialed by strictly-nested anti-pattern profiles, each instance is labelled along two orthogonal difficulty axes, intrinsic (problem size) and incidental (presentation at fixed semantics), and contamination is resisted by minting fresh variants from a private held-out seed. We give construct-validity evidence that the quality order moves established complexity and readability metrics, and report baselines on a four-model open ladder: exact match rises with scale, and the intrinsic knob collapses arithmetic-aggregation accuracy of even the strongest model to zero. Further, development-set scores equal freshly re-minted held-out counterparts within $|Δ|\le 0.012$ (comprehension) and $\le 0.011$ (refactoring); on identical programs, refactoring equivalence ($0.73 \rightarrow 0.99$) is scale-invariant while output prediction collapses; and ablating the generator's self-annotations shows they inflate the weakest model an order of magnitude more than the strongest ($-0.173$ vs $-0.017$): the annotated ladder resolves one of three adjacent pairs where the unannotated resolves all three. Open source (MIT), dependency-free, archived under a persistent DOI.

代码生成数据集构建抗污染多语言

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。