提出新框架，高效检测修复文本转SQL的错误
Understanding, Detecting, and Repairing Real-World In-Context-Learning-Based Text-to-SQL Errors
- 构建首个系统性分析框架，识别27类错误
- 修复成功率提升13.8%，延迟降低67.4%
- 适合需高精度数据库查询的应用场景
大型语言模型（LLM）在文本转SQL任务中利用上下文学习（ICL）能力将自然语言问题转换为SQL查询，但该技术存在正确性问题。本文首次全面研究基于ICL的文本转SQL错误，涵盖四种代表性方法、五种基础修复技术、两个基准数据集和两种LLM设置。研究发现错误普遍存在，总结出7大类27种错误类型。同时发现现有修复方法改进有限，计算开销高且误修率高。基于此，提出MapleDoctor框架，用于错误检测与修复。评估显示，其相比现有方案能修复多13.8%的查询，误修极少，修复延迟降低67.4%。相关代码已公开于GitHub。
原文摘要 · Abstract (English)
Large language models (LLMs) have been adopted for text-to-SQL tasks, utilizing their in-context learning (ICL) capability to translate natural language questions into SQL queries. However, such a technique faces correctness problems. In this paper, we conduct the first comprehensive study of text-to-SQL errors of ICL-based techniques. Our study covers four representative ICL-based techniques, five basic repairing methods, two benchmarks, and two LLM settings. We find that text-to-SQL errors are widespread and summarize 27 error types of 7 categories. We also find that existing repairing attempts have limited correctness improvement while having high computational overhead and many mis-repairs. Based on these findings, we propose MapleDoctor, a novel text-to-SQL error detection and repairing framework. The evaluation demonstrates that MapleDoctor outperforms existing solutions by repairing 13.8% more queries with a negligible number of mis-repairs and reducing 67.4% repair latency. The artifact is publicly available at GitHub.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。