设计专用提示模板，让大模型生成更可靠代码。
Prompt engineering and framework: implementation to increase code reliability based guideline for LLMs
- 用定制提示模板提升大模型写Python代码的准确性。
- 在HumanEval上通过率比零样本和思维链方法高，且节省资源。
- 适合需要高效生成可靠代码的开发者或自动化编程场景。
本文提出一种新型提示方法，旨在提升大型语言模型（LLMs）生成准确Python代码的能力。具体而言，我们设计了一种提示模板，以改善生成代码片段的质量与正确性，使其能够通过测试并产生可靠结果。在两个前沿LLM上基于HumanEval数据集的实验表明，该方法在Pass@k指标上优于广泛研究的零样本和思维链（CoT）方法。此外，相比CoT方法，本方法显著减少了令牌使用量，兼具高效性与资源节约性，降低了计算开销并优化了大模型应用的环境影响。这些发现表明，定制化提示策略可有效提升代码生成性能，为人工智能驱动的编程任务拓展更广泛应用前景。
原文摘要 · Abstract (English)
In this paper, we propose a novel prompting approach aimed at enhancing the ability of Large Language Models (LLMs) to generate accurate Python code. Specifically, we introduce a prompt template designed to improve the quality and correctness of generated code snippets, enabling them to pass tests and produce reliable results. Through experiments conducted on two state-of-the-art LLMs using the HumanEval dataset, we demonstrate that our approach outperforms widely studied zero-shot and Chain-of-Thought (CoT) methods in terms of the Pass@k metric. Furthermore, our method achieves these improvements with significantly reduced token usage compared to the CoT approach, making it both effective and resource-efficient, thereby lowering the computational demands and improving the eco-footprint of LLM capabilities. These findings highlight the potential of tailored prompting strategies to optimize code generation performance, paving the way for broader applications in AI-driven programming tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。