用SQLite实现关键词、语义和混合搜索，轻量高效。
SQLite is Enough. Lexical, Semantic, and Hybrid Search with scrydb
- 基于FTS5和sqlite-vec扩展，支持三种检索方式
- 在多个数据集上验证了混合排序的有效性
- 适合需要轻量级检索的下游应用和智能代理
本文提出scrydb，一个Python库，可在SQLite中实现关键词、语义及混合搜索。关键词搜索利用SQLite的FTS5全文检索扩展；语义搜索基于sqlite-vec向量检索扩展；该库还支持对检索结果进行重排与融合，结合两种方法优势，为信息检索或智能代理任务提供轻量解决方案。我们在多个IR基准数据集上评估scrydb，证明其在基于关键词匹配、语义相似性以及排序融合方面的有效性。同时分析了查询延迟，并揭示效率与效果之间的权衡。scrydb采用MIT许可证开源。
原文摘要 · Abstract (English)
This work introduces scrydb, a Python library that enables lexical, semantic, and hybrid search within SQLite. For lexical search, scrydb leverages SQLite's full-text search extension FTS5. Semantic search builds on sqlite-vec, a SQLite extension for vector search. Furthermore, the library allows users to rerank and fuse retrieval results to combine both lexical and semantic approaches, providing a lightweight solution for downstream tasks in information retrieval (IR) or agentic search. We evaluate scrydb on various IR benchmark datasets and demonstrate its effectiveness in text retrieval based on keyword matching, semantic similarity, and rank fusion. In addition, we provide insights into query latency and the trade-off between efficiency and effectiveness. scrydb is available under the MIT license.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。