arXiv:2507.10182cs.SEcs.AI2025-07被引 1

小模型经微调可高效生成高质量代码后置条件

Breaking the Myth: Can Small Models Infer Postconditions Too?

  • 用70亿参数小模型在定制数据集上微调
  • 在Defects4J上表现媲美甚至超越GPT-4o等大模型
  • 适合资源有限但需自动化验证的开发团队

形式化规范对保障软件正确性至关重要,但手动编写耗时易错。大型语言模型(LLMs)虽能在自然语言意图下生成规范,但其庞大体量和高算力需求引发根本疑问:是否真需大模型?本文证明,经微调的小型语言模型可在极低计算成本下实现高质量后置条件生成。我们构建了包含提示、推理日志与后置条件的专用数据集,并对一个70亿参数的代码模型进行监督微调。该方法处理真实仓库依赖关系并保留前状态信息,生成表达丰富且准确的规范。在真实Java缺陷数据集Defects4J上的评估显示,其在语法正确性、语义正确性及区分缺陷能力方面均达到或超过专有大模型(如GPT-4o)与开源大模型的表现。结果表明,针对特定任务在小规模数据集上微调,即可使小模型达成以往仅靠巨量资源大模型才能实现的效果,为自动化规范生成提供了实用高效的落地路径。

原文摘要 · Abstract (English)

Formal specifications are essential for ensuring software correctness, yet manually writing them is tedious and error-prone. Large Language Models (LLMs) have shown promise in generating such specifications from natural language intents, but the giant model size and high computational demands raise a fundamental question: Do we really need large models for this task? In this paper, we show that a small, fine-tuned language model can achieve high-quality postcondition generation with much lower computational costs. We construct a specialized dataset of prompts, reasoning logs, and postconditions, then supervise the fine-tuning of a $7$B-parameter code model. Our approach tackles real-world repository dependencies and preserves pre-state information, allowing for expressive and accurate specifications. We evaluate the model on a benchmark of real-world Java bugs (Defects4J) and compare against both proprietary giants (e.g., GPT-4o) and open-source large models. Empirical results demonstrate that our compact model matches or outperforms significantly larger counterparts in syntax correctness, semantic correctness, and bug-distinguishing capability. These findings highlight that targeted fine-tuning on a modest dataset can enable small models to achieve results formerly seen only in massive, resource-heavy LLMs, offering a practical and efficient path for the real-world adoption of automated specification generation.

代码生成小模型形式化验证微调

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。