用简单提示策略让大模型生成符合单例模式的代码。
Strategies for Guiding LLMs to Use Software Design Patterns: A Case of Singleton

- 测试13个大模型在4种提示策略下的表现。
- 二元反馈使代码功能提升34.1个百分点,单例匹配率达99.2%。
- 适合想提升生成代码规范性的开发者使用。
大型语言模型(LLMs)能从自然语言提示生成功能性代码,但往往无法稳定遵循高层架构或设计模式。随着LLMs在软件工程中的广泛应用,其生成代码是否符合既定设计原则,对软件长期成功至关重要。本文旨在识别引导LLMs在生成代码中融入设计模式的策略。我们设计了一项计算实验,评估13个LLMs在164个来自HumanEval-X的Java编码挑战中,使用四种提示策略(指令、二元自动反馈、广泛自动反馈、带少样本提示的广泛反馈)生成遵循单例设计模式代码的能力。结果表明,最优策略高度依赖模型类型。总体而言,迭代式二元反馈在保持或提升代码功能的同时,提供了最佳的单例对齐效果。使用指令提示,Llama 3.3在所有案例中均生成单例类,并使通过测试数提升34.1个百分点。采用指令与二元反馈结合,同样取得优异表现。Qwen 3 (8B) 在二元反馈下,单例对齐率达99.2%,功能通过率提升至58.6%。结果表明,即使简单策略也能有效引导LLMs使用设计模式。
原文摘要 · Abstract (English)
Large Language Models (LLMs) can generate functional source code from natural-language prompts, but often fail to consistently follow higher-level architectural structures or design patterns. Since LLMs are increasingly used in software engineering, their ability to apply established design principles to generated code is crucial to the long-term success of software products. Therefore, the goal of this paper is to identify strategies for guiding LLMs to incorporate design patterns into the generated source code. We designed a computational experiment to evaluate the ability of 13 LLMs to generate code that follows the Singleton design pattern, using four prompting strategies: instructions, binary automated feedback, extensive automated feedback, and extensive feedback with few-shot prompts, in 164 Java coding challenges from HumanEval-X. Our results shows that the optimal strategy to guide LLMs to include design patterns depends heavily on the type of model. Still, overall, iterative binary feedback provides the best alignment with Singleton while preserving or improving the code's functionality. With guiding with instructions, Llama 3.3 generated Singleton classes in 100% of cases and improved code functionality, increasing the number of tests passed by 34.1 percentage points. It achieved a similar result with guidance through instructions and binary feedback. Qwen 3 (8B) increased the alignment with Singleton to 99.2% and the functionality to 58.6% using binary feedback. Our result suggests that even simple strategies can be used to guide LLMs to use design patterns.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。