提升文本转SQL的鲁棒性，应对对抗性干扰
Solid-SQL: Enhanced Schema-linking based In-context Learning for Robust Text-to-SQL
- 用LLM增强数据增广训练鲁棒的模式链接模型
- 采用两轮结构相似度检索，提升上下文学习效果
- 在多个扰动数据集上平均提升11.6%，适合实际部署
大型语言模型（LLMs）显著提升了文本转SQL系统的性能，但现有先进方法忽视了系统鲁棒性。实验表明，尽管这些方法在标准数据集上表现优异，但在面对对抗性扰动时准确率明显下降。为此，我们提出Solid-SQL，一种可集成于多种LLMs的鲁棒文本转SQL解决方案。核心在于预处理阶段：通过LLM增强的数据增广训练鲁棒的模式链接模型；并设计两轮、基于结构相似性的示例检索策略，用于上下文学习。该方法在通用Spider和Bird基准上分别达到82.1%和58.9%的SOTA SQL执行准确率。在扰动数据集Spider-Syn、Spider-Realistic和Dr. Spider上，相较基线平均提升11.6%。
原文摘要 · Abstract (English)
Recently, large language models (LLMs) have significantly improved the performance of text-to-SQL systems. Nevertheless, many state-of-the-art (SOTA) approaches have overlooked the critical aspect of system robustness. Our experiments reveal that while LLM-driven methods excel on standard datasets, their accuracy is notably compromised when faced with adversarial perturbations. To address this challenge, we propose a robust text-to-SQL solution, called Solid-SQL, designed to integrate with various LLMs. We focus on the pre-processing stage, training a robust schema-linking model enhanced by LLM-based data augmentation. Additionally, we design a two-round, structural similarity-based example retrieval strategy for in-context learning. Our method achieves SOTA SQL execution accuracy levels of 82.1% and 58.9% on the general Spider and Bird benchmarks, respectively. Furthermore, experimental results show that Solid-SQL delivers an average improvement of 11.6% compared to baselines on the perturbed Spider-Syn, Spider-Realistic, and Dr. Spider benchmarks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。