通过结构对齐评估数据匹配度，提升文本转SQL的微调效果。
Do LLMs Align with My Task? Evaluating Text-to-SQL via Dataset Alignment
- 用结构特征分布比较训练数据与目标数据的匹配度。
- 高对齐时微调后准确率显著提升，低对齐则效果微弱。
- 适合关注跨领域NL2SQL泛化能力的研究者使用。
监督微调（SFT）是适配大语言模型（LLMs）于下游任务的有效方法，但训练数据的差异会影响模型在不同领域的泛化能力。本文研究自然语言转SQL（NL2SQL）任务中的数据集对齐问题，分析训练数据与目标查询在结构特征上的匹配程度及其对模型性能的影响。我们假设可通过比较训练集、目标数据及模型预微调预测结果中结构化SQL特征的分布，来准确估计对齐程度。在三个大规模跨领域NL2SQL基准上，对多个模型家族进行综合实验表明，结构对齐是微调成功的重要预测指标：对齐度高时，微调带来显著的准确率与生成质量提升；对齐度低时，改进几乎不存在。研究强调了在微调中进行对齐感知数据选择的重要性，以实现更有效的跨域泛化。
原文摘要 · Abstract (English)
Supervised Fine-Tuning (SFT) is an effective method for adapting Large Language Models (LLMs) on downstream tasks. However, variability in training data can hinder a model's ability to generalize across domains. This paper studies the problem of dataset alignment for Natural Language to SQL (NL2SQL or text to SQL), examining how well SFT training data matches the structural characteristics of target queries and how this alignment impacts model performance. We hypothesize that alignment can be accurately estimated by comparing the distributions of structural SQL features across the training set, target data, and the model's predictions prior to SFT. Through comprehensive experiments on three large cross-domain NL2SQL benchmarks and multiple model families, we show that structural alignment is a strong predictor of fine-tuning success. When alignment is high, SFT yields substantial gains in accuracy and SQL generation quality; when alignment is low, improvements are marginal or absent. These findings highlight the importance of alignment-aware data selection for effective fine-tuning and generalization in NL2SQL tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。