用代码实体信息选例，提升开源模型的代码解释效果。
Selective Shot Learning for Code Explanation
- 基于代码实体信息选择少样本示例，兼顾语法结构。
- 在两个数据集上优于现有方法，显著提升解释准确率。
- 首个针对开源代码大模型的少样本学习系统评估。
代码解释对软件工程至关重要，帮助开发者高效理解代码功能。近期研究表明，智能选择少量示例能显著提升大语言模型（LLM）的代码解释性能。当前主流的少样本学习（SSL）方法包括基于标记和基于嵌入的方法，但这些方法多在专有模型上评估，未充分探索开源代码大模型（Code-LLMs）。此外，现有方法忽视编程语言语法特征。为此，我们开展对比研究并提出一种新方法SSL_ner，利用代码实体信息进行示例选择。实验揭示多个关键洞察，并证明SSL_ner在两个数据集上均优于现有方法。据我们所知，这是首个系统性评估开源代码大模型在代码解释任务中少样本选择策略的基准研究。
原文摘要 · Abstract (English)
Code explanation plays a crucial role in the software engineering domain, aiding developers in grasping code functionality efficiently. Recent work shows that the performance of LLMs for code explanation improves in a few-shot setting, especially when the few-shot examples are selected intelligently. State-of-the-art approaches for such Selective Shot Learning (SSL) include token-based and embedding-based methods. However, these SSL approaches have been evaluated on proprietary LLMs, without much exploration on open-source Code-LLMs. Additionally, these methods lack consideration for programming language syntax. To bridge these gaps, we present a comparative study and propose a novel SSL method (SSL_ner) that utilizes entity information for few-shot example selection. We present several insights and show the effectiveness of SSL_ner approach over state-of-the-art methods across two datasets. To the best of our knowledge, this is the first systematic benchmarking of open-source Code-LLMs while assessing the performances of the various few-shot examples selection approaches for the code explanation task.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。