让小模型高效执行带类型约束的LLM函数，提升查询准确率与速度
Play by the Type Rules: Inferring Constraints for LLM Functions in Declarative Programs
- 用小型开源模型直接解析并执行SQL风格函数
- 多跳问答任务准确率提升7%，延迟降低53%
- 适合需要高效类型安全的数据库+LLM集成场景
在声明式查询语言中集成基于大语言模型的算子，可结合廉价可解释函数与强大通用推理能力。但为发挥数据库查询语言（如SQL）的优化执行优势，生成结果必须符合类型检查器与数据库内容的约束。现有方法依赖大量基于LLM的后处理调用以确保对齐，带来性能瓶颈。我们研究了不同规模开源模型在解析和执行基于SQL的查询语言函数中的能力，发现小型模型在混合数据源上作为函数执行器表现优异。为此，我们提出一种高效方法以保证LLM函数的类型正确性，在多跳问答数据集上实现7%的准确率提升，相比同类方案延迟降低53%。代码已开源：https://github.com/parkervg/blendsql
原文摘要 · Abstract (English)
Integrating LLM powered operators in declarative query languages allows for the combination of cheap and interpretable functions with powerful, generalizable language model reasoning. However, in order to benefit from the optimized execution of a database query language like SQL, generated outputs must align with the rules enforced by both type checkers and database contents. Current approaches address this challenge with orchestrations consisting of many LLM-based post-processing calls to ensure alignment between generated outputs and database values, introducing performance bottlenecks. We perform a study on the ability of various sized open-source language models to both parse and execute functions within a query language based on SQL, showing that small language models can excel as function executors over hybrid data sources. Then, we propose an efficient solution to enforce the well-typedness of LLM functions, demonstrating 7% accuracy improvement on a multi-hop question answering dataset with 53% improvement in latency over comparable solutions. We make our implementation available at https://github.com/parkervg/blendsql
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。