arXiv:2605.10090cs.IR2026-05中稿 · ICDE'26

针对多核CPU向量搜索,提出芯片级负载感知调度框架,显著提升吞吐与降低延迟。

CCD-Level and Load-Aware Thread Orchestration for In-Memory Vector ANNS on Multi-Core CPUs

  • 基于CCD架构设计芯片级线程调度,融合查询间与查询内并行
  • 在真实业务中实现最高3.7倍吞吐提升,尾延迟降低30%-90%
  • 适合大规模在线向量搜索系统,如推荐、广告与搜索引擎

向量近似最近邻搜索(ANNS)支撑搜索引擎、推荐系统和广告服务。近期索引技术使CPU成为百万级内存向量搜索的低成本选择,但每核吞吐受限于向量读取的内存延迟和距离计算的计算强度。随着业务规模扩大与硬件演进,现代基于CCD的多核CPU已在生产环境广泛应用。然而,单纯增加核心数无法获得最优性能扩展。我们分析了生产环境中真实请求分布,发现向量搜索具有高访问局部性与低缓存利用率,根源在于忽视了CCD架构的多芯片小片特性。为此,我们提出一种面向工作负载与硬件的芯片级线程调度框架:(i) 统一支持跨查询并行的HNSW搜索与查询内并行的IVF搜索;(ii) 实现缓存友好且自适应的工作负载任务映射;(iii) 采用芯片级任务窃取缓解负载不均。在小红书(RedNote)搜索、推荐与广告的真实业务负载上应用,该方法最高实现3.7倍吞吐提升,P50与P999延迟降低30%-90%。具体而言,缓存未命中率下降6%-30%,总CPU停顿时间减少20%-80%。

原文摘要 · Abstract (English)

Vector approximate nearest neighbor search (ANNS) underpins search engines, recommendation systems, and advertising services. Recent advances in ANNS indexes make CPU a cost-effective choice for serving million-scale, in-memory vector search, yet per-core throughput remains constrained by memory access latency of vector reading and the compute intensity of distance evaluations in production deployments. With the growing scale of the business and advances in hardware, modern CCD-based multi-core CPUs have been widely deployed for high throughput in our services. However, we find that simply increasing core counts does not yield optimal performance scaling. To improve the efficiency of more cores from the CCD-based architecture, we analyze the distributions of real-world requests in our production environments. We observe high access locality in vector search in our online services and low cache utilization, resulting from overlooking the multi-chiplet nature of CCD based CPUs. Hence, we propose a workload- and hardware-aware thread orchestration framework at CCD-level that (i) provides a uniform interface for both inter-query parallel HNSW search and intra-query parallel IVF search, (ii) achieves cache-friendly and workload-adaptive mapping of task dispatching, and (iii) employs CCD-aware task stealing to address load imbalance. Applied to real production workloads from search, recommendation, and advertising services of Xiaohongshu (RedNote), our approach delivers up to 3.7x higher throughput and 30-90% reductions in P50 and P999 latency. In detail, compared with the original framework, the cache-miss ratio decreases by 6-30%, and the total CPU stall is reduced by 20-80%.

向量搜索多核调度CCD架构低延迟

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。