通过动态少样本与一致性对齐,提升文本转SQL的准确率与稳定性。
OpenSearch-SQL: Enhancing Text-to-SQL with Dynamic Few-shot and Consistency Alignment
- 分四模块处理文本转SQL,引入一致性对齐减少幻觉和指令偏离。
- 在BIRD数据集上达69.3%执行准确率,测试集72.28%,指标领先。
- 无需微调基础模型,框架轻量可复用,适合实际部署场景。
尽管多智能体协作的大语言模型在文本转SQL任务中取得显著进展,但其性能仍受框架不完整、指令遵循失败和模型幻觉等问题制约。为此,我们提出OpenSearch-SQL,将文本转SQL任务分解为预处理、提取、生成与优化四个模块,并引入基于一致性对齐机制的对齐模块。该架构通过模块间输入输出对齐,有效降低指令偏离与幻觉问题。同时,设计了名为SQL-Like的中间语言,并基于此优化结构化思维链(CoT)。此外,提出自学习查询-思维链-SQL的动态少样本策略。实验表明,直接使用未微调的基础模型,OpenSearch-SQL在BIRD开发集上实现69.3%的执行准确率(EX),测试集达72.28%,奖励有效性评分(R-VES)为69.36%,三项指标均位居提交时前列,展现出高效与高精度的综合优势。
原文摘要 · Abstract (English)
Although multi-agent collaborative Large Language Models (LLMs) have achieved significant breakthroughs in the Text-to-SQL task, their performance is still constrained by various factors. These factors include the incompleteness of the framework, failure to follow instructions, and model hallucination problems. To address these problems, we propose OpenSearch-SQL, which divides the Text-to-SQL task into four main modules: Preprocessing, Extraction, Generation, and Refinement, along with an Alignment module based on a consistency alignment mechanism. This architecture aligns the inputs and outputs of agents through the Alignment module, reducing failures in instruction following and hallucination. Additionally, we designed an intermediate language called SQL-Like and optimized the structured CoT based on SQL-Like. Meanwhile, we developed a dynamic few-shot strategy in the form of self-taught Query-CoT-SQL. These methods have significantly improved the performance of LLMs in the Text-to-SQL task. In terms of model selection, we directly applied the base LLMs without any post-training, thereby simplifying the task chain and enhancing the framework's portability. Experimental results show that OpenSearch-SQL achieves an execution accuracy(EX) of 69.3% on the BIRD development set, 72.28% on the test set, and a reward-based validity efficiency score (R-VES) of 69.36%, with all three metrics ranking first at the time of submission. These results demonstrate the comprehensive advantages of the proposed method in both effectiveness and efficiency.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。