arXiv:2511.04153cs.CLcs.AI2025-11被引 1

对比三种智能体流水线,提升小模型文本转SQL准确率。

BAPPA: Benchmarking Agents, Plans, and Pipelines for Automated Text-to-SQL Generation

  • 用多个智能体协作讨论、规划或聚合生成SQL,提升准确性。
  • 小模型经三轮讨论后执行准确率最高提升10.6%。
  • 适合想用小模型高效做文本转SQL的研究者和开发者。

文本转SQL系统为普通人访问数据库提供了自然语言接口。然而,现有大语言模型在面对复杂模式和推理时,难以从自然指令生成准确的SQL。以往研究多聚焦于使用顶级模型的复杂、不切实际的流水线,而小型高效模型常被忽视。本文探索三种多智能体流水线,在从小到大的开源模型上进行系统性性能测试:(1)多智能体讨论流水线,智能体迭代批判并优化SQL查询,由裁判智能体合成最终答案;(2)规划-编码流水线,思考型规划器生成分步生成计划,编码器合成查询;(3)编码-聚合流水线,多个编码器独立生成查询,由推理智能体选择最优结果。在Bird-Bench Mini-Dev数据集上的实验表明,多智能体讨论可显著提升小模型性能,如Qwen2.5-7b-Instruct在三轮讨论后执行准确率提升10.6%。三种流水线中,LLM Reasoner-Coder表现最佳,使用DeepSeek-R1-32B与QwQ-32B作为规划器,使Gemma 3 27B IT的准确率从52.4%提升至最高56.4%。代码已开源:https://github.com/treeDweller98/bappa-sql。

原文摘要 · Abstract (English)

Text-to-SQL systems provide a natural language interface that can enable even laymen to access information stored in databases. However, existing Large Language Models (LLM) struggle with SQL generation from natural instructions due to large schema sizes and complex reasoning. Prior work often focuses on complex, somewhat impractical pipelines using flagship models, while smaller, efficient models remain overlooked. In this work, we explore three multi-agent LLM pipelines, with systematic performance benchmarking across a range of small to large open-source models: (1) Multi-agent discussion pipeline, where agents iteratively critique and refine SQL queries, and a judge synthesizes the final answer; (2) Planner-Coder pipeline, where a thinking model planner generates stepwise SQL generation plans and a coder synthesizes queries; and (3) Coder-Aggregator pipeline, where multiple coders independently generate SQL queries, and a reasoning agent selects the best query. Experiments on the Bird-Bench Mini-Dev set reveal that Multi-Agent discussion can improve small model performance, with up to 10.6% increase in Execution Accuracy for Qwen2.5-7b-Instruct seen after three rounds of discussion. Among the pipelines, the LLM Reasoner-Coder pipeline yields the best results, with DeepSeek-R1-32B and QwQ-32B planners boosting Gemma 3 27B IT accuracy from 52.4% to the highest score of 56.4%. Codes are available at https://github.com/treeDweller98/bappa-sql.

文本转SQL多智能体小模型优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。