D-NOVA通过硬件软件协同设计,在存储器内实现高效向量检索,显著降低延迟与能耗。
D-NOVA: In-Storage Retrieval Accelerator via Dual-Bound 3D NAND-Optimized Similarity Search with Vector Adaptation

- 将相似度搜索直接嵌入NAND闪存阵列,利用双界紧致感知距离度量提升效率。
- 相比CPU基线快41.7倍、能效高71倍,吞吐量达现有存储级加速器的12.13倍。
- 适合需要低延迟、高能效的大型语言模型外部知识检索场景。
检索增强生成(RAG)通过从外部知识库中检索相关信息来增强大语言模型推理的事实准确性。然而,其密集向量检索引入了显著的延迟和能耗开销,成为主要性能瓶颈。尽管近期的存储内加速器旨在减少数据搬移,但仍依赖于内存外的主机或嵌入式处理器,其中近70%的总检索时间消耗于此,无法彻底克服带宽限制,导致新的内存瓶颈。为此,我们提出D-NOVA,一种软硬件协同设计的存储内检索加速器。D-NOVA通过深度嵌入倒排文件(IVF)层级检索流水线至NAND存储阵列,实现搜索功能的原位执行。该设计引入一种专为NAND串优化的新距离度量——双界紧致相似度感知(DTS),并搭配轻量级对比适配器,将嵌入向量映射至适配DTS的域,恢复接近软件的召回率,同时提升性能与能效。D-NOVA相较CPU基线最高提速41.7倍、能效提升71倍,吞吐量达同类存储级加速器的12.13倍,且能效最高提升1.26倍,验证了全存储内向量搜索在可扩展RAG加速中的潜力。
原文摘要 · Abstract (English)
Retrieval-Augmented Generation (RAG) enhances the factual grounding of large language model (LLM) inference by retrieving relevant information from external knowledge bases. However, its dense vector retrieval introduces significant latency and energy overhead, becoming the primary performance bottleneck. Although recent in-storage accelerators aim to reduce data movement, they still rely on host or embedded processors outside the memory, where nearly 70% of the total retrieval time is spent. As a result, they cannot fully overcome the bandwidth limitations, leading to yet another memory bottleneck. To tackle these limitations, we present D-NOVA, a hardware-software co-designed in-storage retrieval accelerator. D-NOVA executes an inverted file (IVF)-based hierarchical retrieval pipeline by deeply embedding the search functionality directly into the NAND memory array. This is achieved by incorporating a new distance metric, Dual-Bound Tight Similarity Sensing (DTS), which is specifically tailored for searching within the NAND string. In addition, we introduce a lightweight contrastive adapter that maps embedding vectors into a DTS-friendly domain, recovering near-software recall while improving performance and energy efficiency. D-NOVA is up to 41.7x faster and 71x more energy-efficient than a CPU baseline, and achieves 12.13x higher throughput while being up to 1.26x more energy-efficient than state-of-the-art in-storage RAG accelerators, demonstrating the potential of fully in-storage vector search for scalable RAG acceleration.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。