对比7种向量数据库，实测性能、精度与资源消耗，指导实际选型。
A Comprehensive Empirical Evaluation of Vector Database Systems for Approximate Nearest Neighbor Search: Performance, Quality, and Resource Trade-offs
- 在6个数据集上系统测试7款主流向量库，覆盖400万+向量
- FAISS单机吞吐最高（866 QPS），Weaviate召回率超99%，LanceDB建索引最快
- 提供可复现的评测框架，适合大模型应用开发者选型参考
向量数据库已成为现代人工智能应用的关键基础设施，尤其在检索增强生成（RAG）、语义搜索和推荐系统中。尽管其重要性日益凸显，但缺乏全面且可复现的基准测试来联合评估检索质量、查询延迟、吞吐量和资源利用率。本文对七款主流向量数据库——FAISS、Qdrant、Milvus、Weaviate、Chroma、pgvector 和 LanceDB 进行系统性实证评估。实验涵盖六个不同数据集，包括经典计算机视觉特征（SIFT、GIST）和基于Transformer的文本嵌入（MS MARCO、GloVe），向量总量超过400万，维度范围为96至960。我们测量了15项指标，涵盖检索质量（Recall@K、Precision@K、MRR、NDCG@K、Hit Rate@K）、查询性能（延迟分位数、QPS、冷启动延迟）以及资源消耗（索引构建时间、内存、存储）。在SIFT1M数据集上，FAISS实现最高单节点吞吐量（866 QPS），但缺少数据库运营功能；Weaviate提供最佳开箱即用召回率（>99%）；Qdrant在全功能数据库中延迟最低（中位数4.55~ms）；LanceDB以牺牲部分检索质量为代价，实现显著更快的索引构建速度。研究为实践者提供了系统选型指南，并开源了评测框架。
原文摘要 · Abstract (English)
Vector databases have emerged as critical infrastructure for modern artificial intelligence applications, particularly retrieval-augmented generation (RAG), semantic search, and recommendation systems. Despite their growing importance, there remains a significant gap in comprehensive, reproducible benchmarks that jointly evaluate retrieval quality, query latency, throughput, and resource utilization. We present a systematic empirical evaluation of seven prominent vector database systems: FAISS, Qdrant, Milvus, Weaviate, Chroma, pgvector, and LanceDB. Our methodology spans six diverse datasets, from classical computer-vision descriptors (SIFT, GIST) to transformer-based text embeddings (MS MARCO, GloVe), encompassing over 4 million vectors at dimensionalities from 96 to 960. We measure 15 metrics spanning retrieval quality (Recall@K, Precision@K, MRR, NDCG@K, Hit Rate@K), query performance (latency percentiles, QPS, cold-start latency), and resource consumption (index build time, memory, storage). On SIFT1M, FAISS achieves the highest single-node throughput (866 QPS) but lacks database operational features; Weaviate provides the best out-of-the-box recall (> 99%); Qdrant offers the best latency among full databases (4.55~ms median); and LanceDB trades retrieval quality for substantially faster index construction. We derive system-selection guidelines for practitioners and release our benchmarking framework as open-source software.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。