用轻量代理模型加速AI查询，成本降低100倍以上。
100x Cost & Latency Reduction: Performance Analysis of AI Query Approximation using Lightweight Proxy Models
- 用嵌入向量的轻量代理模型替代复杂LLM进行语义过滤和排序。
- 在百万级数据上实现超100倍成本与延迟降低，准确率不降反升。
- 适合需要实时分析的数据库场景，尤其适合大规模在线查询。
近年来，多个数据仓库和数据库厂商推出了名为AI Query的SQL扩展功能，允许用户在SQL中指定由大语言模型（LLM）评估的函数与条件，从而显著拓展了结构化与非结构化数据结合查询的能力。尽管LLM具备强大的语义推理能力，但当其被调用数千次时，成本与延迟会急剧上升。本文对一种近期提出的AI查询近似方法进行了全面评估，该方法通过使用低成本且高精度的代理模型处理嵌入向量，实现了对语义过滤算子超过100倍的成本与延迟降低，并在语义排序任务中也取得显著性能提升。实验表明，尽管大幅降低了资源开销，代理模型仍保持甚至在某些情况下提升了准确率，涵盖包含1000万行数据的扩展亚马逊评论基准数据集。本文还提出了两种架构：一是在Google BigQuery中支持纯在线查询的OLAP友好架构；二是在AlloyDB中支持低延迟混合事务/分析处理（HTAP）的架构，通过将代理模型训练移至离线阶段进一步降低延迟，并提供了加速训练的技术方案。
原文摘要 · Abstract (English)
Several data warehouse and database providers have recently introduced extensions to SQL called AI Queries, enabling users to specify functions and conditions in SQL that are evaluated by LLMs, thereby broadening significantly the kinds of queries one can express over the combination of structured and unstructured data. LLMs offer remarkable semantic reasoning capabilities, making them an essential tool for complex and nuanced queries that blend structured and unstructured data. While extremely powerful, these AI queries can become prohibitively costly when invoked thousands of times. This paper provides an extensive evaluation of a recent AI query approximation approach that enables low cost analytics and database applications to benefit from AI queries. The approach delivers >100x cost and latency reduction for the semantic filter operator and also important gains for semantic ranking. The cost and performance gains come from utilizing cheap and accurate proxy models over embedding vectors. We show that despite the massive gains in latency and cost, these proxy models preserve accuracy and occasionally improve accuracy across various benchmark datasets, including the extended Amazon reviews benchmark that has 10M rows. We present an OLAP-friendly architecture within Google BigQuery for this approach for purely online (ad hoc) queries, and a low-latency HTAP database-friendly architecture in AlloyDB that could further improve the latency by moving the proxy model training offline. We present techniques that accelerate the proxy model training.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。