用自解释关键词提升大模型代码生成准确率
Self-Explained Keywords Empower Large Language Models for Code Generation
- 让大模型自己提取并解释问题中的关键术语，按频率排序
- 在HumanEval上将代码通过率从85.4%提升至93.3%
- 特别适合处理低频术语导致的生成错误问题
大语言模型（LLMs）在代码生成任务中表现优异，但由于训练数据存在长尾分布，低频术语在训练中常被忽视。这导致模型在生成代码时容易误解或遗漏问题中的特定低频关键词，影响生成结果准确性。为此，本文提出一种名为SEK（Self-Explained Keywords）的新方法：由大模型自身提取并解释问题描述中的关键词，并根据其出现频率进行排序。在HumanEval(+), MBPP(+), APPS三个基准上的实验表明，SEK显著提升了五种代表性大模型的代码生成能力，效果稳定且显著。例如，在HumanEval基准上，DeepSeek-Coder-V2-Instruct的Pass@1从85.4%提升至93.3%。进一步分析证实，该方法使模型注意力从低频关键词转向高频对应词。
原文摘要 · Abstract (English)
Large language models (LLMs) have achieved impressive performance in code generation. However, due to the long-tail distribution of LLMs' training data, low-frequency terms are typically underrepresented in the training process. Consequently, LLMs often misunderstand or overlook problem-specific, low-frequency keywords during code generation, compromising the accuracy of the generated code. To address this, we propose a novel technique named SEK(\textbf{S}elf-\textbf{E}xplained \textbf{K}eywords), which empowers an LLM for better code generation by extracting and explaining the key terms in the problem description with the LLM itself and ranking them based on frequency. Comprehensive experiments across three benchmarks, i.e., HumanEval(+), MBPP(+), and APPS, with five representative LLMs, show that SEK can significantly improve LLMs in code generation, yielding substantial and consistent gains. For instance, SEK improves the Pass@1 of DeepSeek-Coder-V2-Instruct from 85.4\% to 93.3\% on the Humaneval benchmark. Further analysis confirms that SEK enables the LLMs to shift their attention from low-frequency keywords to their corresponding high-frequency counterparts.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。