给数据查询智能体注入经验性知识，纠正其对数据库的误解。
Arming Data Agents with Tribal Knowledge
- 通过分析智能体错误生成修正知识，弥补其对字段含义的误解。
- 在Spider 2.0上提升准确率最高达16.9%，BIRD上提升13.7%。
- 适用于希望提升复杂数据库查询准确率的研究者与开发者。
自然语言转SQL（NL2SQL）使非专家用户可通过自然语言查询关系型数据库。近期基于大语言模型（LLMs）推理能力的NL2SQL智能体显著提升了翻译性能，但在面对大规模真实数据库时仍会出错，因其缺乏正确使用底层数据的知识（如字段意图），并在查询时形成误判。已有研究尝试生成数据库事实以提供上下文，但仅复述数据内容，未解决智能体的误判问题。本文提出Tk-Boost，一种可集成到任意NL2SQL智能体的增强框架，引入‘部落知识’——通过使用数据库积累的经验性知识，纠正智能体的误判。Tk-Boost首先让智能体在数据库上回答若干查询，通过分析其错误识别误判，并生成针对性知识；这些知识按适用条件索引，确保精准检索。在新查询中，系统利用该知识反馈给智能体，在生成SQL时修正误判，从而提升准确性。在BIRD和Spider 2.0基准上的实验表明，Tk-Boost使各类NL2SQL智能体在Spider 2.0上准确率最高提升16.9%，在BIRD上提升13.7%。
原文摘要 · Abstract (English)
Natural language to SQL (NL2SQL) translation enables non-expert users to query relational databases through natural language. Recently, NL2SQL agents, powered by the reasoning capabilities of Large Language Models (LLMs), have significantly advanced NL2SQL translation. Nonetheless, NL2SQL agents still make mistakes when faced with large-scale real-world databases because they lack knowledge of how to correctly leverage the underlying data (e.g., knowledge about the intent of each column) and form misconceptions about the data when querying it, leading to errors. Prior work has studied generating facts about the database to provide more context to NL2SQL agents, but such approaches simply restate database contents without addressing the agent's misconceptions. In this paper, we propose Tk-Boost, a bolt-on framework for augmenting any NL2SQL agent with tribal knowledge: knowledge that corrects the agent's misconceptions in querying the database accumulated through experience using the database. To accumulate experience, Tk-Boost first asks the NL2SQL agent to answer a few queries on the database, identifies the agent's misconceptions by analyzing its mistakes on the database, and generates tribal knowledge to address them. To enable accurate retrieval, Tk-Boost indexes this knowledge with applicability conditions that specify the query features for which the knowledge is useful. When answering new queries, Tk-Boost uses this knowledge to provide feedback to the NL2SQL agent, resolving the agent's misconceptions during SQL generation, and thus improving the agent's accuracy. Extensive experiments across the BIRD and Spider 2.0 benchmarks with various NL2SQL agents shows Tk-Boost improves NL2SQL agents accuracy by up to 16.9% on Spider 2.0 and 13.7% on BIRD
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。