让浏览器实现超快近似最近邻搜索，兼顾隐私与低延迟。
WebANNS: Fast and Efficient Approximate Nearest Neighbor Search in Web Browsers
- 用WebAssembly突破浏览器计算瓶颈，提升搜索速度。
- 查询延迟从10秒降至10毫秒，99%分位降低743倍。
- 适合需要本地化、低延迟的AI网页应用开发者使用。
近似最近邻搜索（ANNS）已成为现代AI基础设施的关键组件，尤其在检索增强生成（RAG）应用中至关重要。为保护隐私并适配异构设备部署，众多基于浏览器的ANNS引擎应运而生。然而，浏览器存在计算能力弱、外部存储访问难、内存占用高等挑战，现有最先进方案未能全面解决。本文提出WebANNS，一种专为浏览器设计的新型ANNS引擎。它利用WebAssembly克服计算瓶颈，采用懒加载策略优化外部存储数据读取，通过启发式方法降低内存占用。实验表明，WebANNS在速度与内存效率上表现优异：相较当前最优方案，99%分位查询延迟最高降低743.8倍，内存使用减少达39%。查询时间从10秒缩短至10毫秒级，使浏览器内执行ANNS具备实际可用性。
原文摘要 · Abstract (English)
Approximate nearest neighbor search (ANNS) has become vital to modern AI infrastructure, particularly in retrieval-augmented generation (RAG) applications. Numerous in-browser ANNS engines have emerged to seamlessly integrate with popular LLM-based web applications, while addressing privacy protection and challenges of heterogeneous device deployments. However, web browsers present unique challenges for ANNS, including computational limitations, external storage access issues, and memory utilization constraints, which state-of-the-art (SOTA) solutions fail to address comprehensively. We propose WebANNS, a novel ANNS engine specifically designed for web browsers. WebANNS leverages WebAssembly to overcome computational bottlenecks, designs a lazy loading strategy to optimize data retrieval from external storage, and applies a heuristic approach to reduce memory usage. Experiments show that WebANNS is fast and memory efficient, achieving up to $743.8\times$ improvement in 99th percentile query latency over the SOTA engine, while reducing memory usage by up to 39\%. Note that WebANNS decreases query time from 10 seconds to the 10-millisecond range in browsers, making in-browser ANNS practical with user-acceptable latency.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。