用1比特量化+异构计算,让百亿向量搜索快100倍以上。
Ascend-RaBitQ: Heterogeneous NPU-CPU Acceleration of Billion-Scale Similarity Search with 1-bit Quantization

- NPU做粗排,CPU做精排,各用其长突破性能瓶颈。
- 索引构建速度提升3.0到62.8倍,吞吐最高快11.7倍。
- 专为昇腾架构设计,适合大规模向量检索系统部署。
向量相似性搜索是现代AI系统的关键组件,但传统基于CPU的实现面对百亿级语料库时,受限于计算开销和内存带宽,存在根本性扩展瓶颈。尽管神经网络处理器(NPUs)具备数量级更高的算力密度,现有针对CPU/GPU优化的1比特RaBitQ量化方法因硬件不匹配无法直接移植至NPU架构,且同质化设计难以兼顾精度、内存占用与性能。本文提出Ascend-RaBitQ,首个面向百亿规模向量搜索的异构NPU-CPU优化IVF-RaBitQ系统,核心思想是将粗排序(NPU)与细排序(CPU)解耦,使各阶段可充分发挥硬件优势,打破长期存在的精度-内存-性能权衡。系统采用三阶段异构执行路径:1比特量化向量在AI Core上加速粗排序,设备端AI CPU处理Top-k,主机CPU对全精度向量进行细重排序。引入四项针对昇腾架构的原生优化:融合AIC-AIV操作并行计算距离、重构计算流程以利用旋转正交性、细粒度索引块级负载均衡打破查询边界、以及AI Core与AI CPU间的跨核流水线并行以隐藏Top-k延迟。在标准数据集上的评估表明,Ascend-RaBitQ相比CPU基线索引构建速度提升3.0至62.8倍,吞吐最高达最快CPU IVF-RaBitQ实现的11.7倍,比数学等价的CPU基线快两个数量级以上,并在分布式多NPU系统上展现出良好可扩展性。
原文摘要 · Abstract (English)
Vector similarity search is a critical component of modern AI systems, but traditional CPU-based implementations face fundamental scalability bottlenecks for billion-scale corpora due to prohibitive computational overhead and memory bandwidth limitations. While Neural Processing Units (NPUs) offer orders-of-magnitude higher compute density, existing CPU/GPU-optimized 1-bit RaBitQ quantization implementations cannot be directly ported to NPU architectures due to fundamental hardware mismatches, and homogeneous design paradigms struggle to simultaneously balance accuracy, memory footprint, and performance. This paper presents Ascend-RaBitQ, the first heterogeneous NPU-CPU optimized IVF-RaBitQ system for billion-scale vector search, built on the core insight that decoupling coarse ranking (NPU) from fine ranking (CPU) allows each stage to leverage its optimal hardware, breaking the long-standing accuracy-memory-performance trade-off. We propose a three-stage heterogeneous execution path comprising AI Core-accelerated coarse ranking on 1-bit quantized vectors, on-device AI CPU Top-k processing, and host CPU fine re-ranking on full-precision vectors. We introduce four NPU architecture-native optimizations: fused AIC-AIV operators for parallel distance computation, computation flow restructuring to exploit rotation orthogonality, fine-grained index block-level load balancing that breaks query boundaries, and intra-NPU pipeline parallelism between AI Core and AI CPU to mask Top-k latency. Evaluation on standard datasets shows that Ascend-RaBitQ achieves 3.0X to 62.8X faster index construction than the CPU baseline, up to 11.7X throughput improvement over the fastest CPU IVF-RaBitQ implementation, and over two orders of magnitude over the mathematically equivalent CPU baseline, while demonstrating encouraging scalability on distributed multi-NPU systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。