通过新数据构建与动态掩码微调，显著提升大模型修复SQL错误的能力。
PDC & DM-SFT: A Road for LLM SQL Bug-Fix Enhancing
- 从广度和深度双视角构建渐进式训练数据集
- 动态掩码微调减少训练步数并缓解代码修复中的迷失问题
- 小模型性能超越更大规模的现有最佳模型
代码大语言模型（Code LLM）如Code Llama和DeepSeek-Coder在代码生成任务中表现出色，但多数模型侧重于生成正确代码，而在漏洞修复方面表现不佳。本文提出一套方法以增强大模型的SQL漏洞修复能力，主要包括两部分：从零开始的渐进式数据集构建（PDC）和动态掩码监督微调（DM-SFT）。PDC从广度优先和深度优先两个角度提出两种数据扩展方法；DM-SFT引入一种高效的漏洞修复监督学习方法，显著减少训练步数，并缓解SQL代码修复训练中的“迷失”问题。实验表明，采用这两种方法训练的代码模型，在性能上超越了当前所有更大规模的最优模型。
原文摘要 · Abstract (English)
Code Large Language Models (Code LLMs), such as Code llama and DeepSeek-Coder, have demonstrated exceptional performance in the code generation tasks. However, most existing models focus on the abilities of generating correct code, but often struggle with bug repair. We introduce a suit of methods to enhance LLM's SQL bug-fixing abilities. The methods are mainly consisted of two parts: A Progressive Dataset Construction (PDC) from scratch and Dynamic Mask Supervised Fine-tuning (DM-SFT). PDC proposes two data expansion methods from the perspectives of breadth first and depth first respectively. DM-SFT introduces an efficient bug-fixing supervised learning approach, which effectively reduce the total training steps and mitigate the "disorientation" in SQL code bug-fixing training. In our evaluation, the code LLM models trained with two methods have exceeds all current best performing model which size is much larger.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。