用自动探索+指令生成,让大模型更好理解复杂数据库。
DB-Explore: Automated Database Exploration and Instruction Synthesis for Text-to-SQL
- 构建数据库图谱,用GPT-4挖掘结构与语义模式
- 在BIRD上达67.0%执行准确率,SPIDER上87.8%
- 开源轻量实现,效果超多个GPT-4系统
基于大语言模型(LLM)的文本转SQL系统在自然语言到SQL的转换中表现出色,但面对复杂数据库结构和领域特定查询时仍存在困难,因其主要关注逻辑推理与语法优化,忽视了对数据库的整体理解。为此,我们提出DB-Explore框架,通过自动化探索与指令合成,系统性地将LLM与数据库知识对齐。该框架构建数据库图谱以捕捉复杂关系模式,利用GPT-4系统化挖掘结构规律与语义知识,并生成指令以提炼知识用于高效微调LLM。通过多样化采样策略与自动指令生成,实现对数据库的全面理解,弥合数据库结构与语言模型之间的鸿沟。在SPIDER和BIRD基准上的实验验证了其有效性:在BIRD上达到67.0%的执行准确率,在SPIDER上达到87.8%。值得注意的是,基于Qwen2.5-Coder-7B的开源实现以极低计算成本达到领先性能,优于多个依赖GPT-4的文本转SQL系统。
原文摘要 · Abstract (English)
Recent text-to-SQL systems powered by large language models (LLMs) have demonstrated remarkable performance in translating natural language queries into SQL. However, these systems often struggle with complex database structures and domain-specific queries, as they primarily focus on enhancing logical reasoning and SQL syntax while overlooking the critical need for comprehensive database understanding. To address this limitation, we propose DB-Explore, a novel framework that systematically aligns LLMs with database knowledge through automated exploration and instruction synthesis. DB-Explore constructs database graphs to capture complex relational schemas, leverages GPT-4 to systematically mine structural patterns and semantic knowledge, and synthesizes instructions to distill this knowledge for efficient fine-tuning of LLMs. Our framework enables comprehensive database understanding through diverse sampling strategies and automated instruction generation, bridging the gap between database structures and language models. Experiments conducted on the SPIDER and BIRD benchmarks validate the effectiveness of DB-Explore, achieving an execution accuracy of 67.0% on BIRD and 87.8% on SPIDER. Notably, our open-source implementation based on Qwen2.5-Coder-7B achieves state-of-the-art results at minimal computational cost, outperforming several GPT-4-driven Text-to-SQL systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。