用16GB显存的开源小模型，低成本高效替代大数据库中的闭源语言模型。
Large Databases Need Small, Open-Weight Language Models
- 在16GB显存上运行量化开源模型，本地部署替代闭源API。
- 实验成本降低390倍，延迟减少3.8倍，精度不降反升。
- 适合需控制成本、追求可解释性的数据库智能研究者。
基于专有API的语言模型系统通常采用按令牌计费的模式，在大规模数据库场景下，仅一次实验成本就可能超过1万美元，严重制约研究与实际应用。本文证明，仅需16GB显存的量化开源模型在本地运行，即可在更低延迟和极低价格下达到甚至超越闭源模型的准确率，挑战了闭源API对语言模型-数据库集成的必要性假设。我们提出了在语言模型-数据库系统中高效部署此类模型所需的关键系统优化，并将其集成至BlendSQL v0.1.0框架，实现整体成本下降390倍、延迟降低3.8倍。代码已公开于https://github.com/CapitalOne-Research/play-by-the-type-rules/tree/main/sembench。
原文摘要 · Abstract (English)
Language model systems built around proprietary APIs often operate on a token-based cost model. This becomes prohibitively expensive in the context of large databases, where LM-enhanced relational operators can incur costs exceeding $10,000 for a single set of experiments, hindering thorough research and practical deployment. In this paper, we demonstrate that quantized, open-weight models running locally on just 16GB of VRAM can match or exceed the accuracy of closed-source counterparts at lower latency and a fraction of the price, challenging the prevailing assumption that closed-source LM APIs are necessary for effective LM-database integration. We present and analyze the key system optimizations required to efficiently deploy these open-weight models within an LM-DB system. By integrating these local models into the BlendSQL v0.1.0 framework, we demonstrate a 390x reduction in overall costs and 3.8x reduction in latency compared to a proprietary LM API. We make our code available at https://github.com/CapitalOne-Research/play-by-the-type-rules/tree/main/sembench.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。