arXiv:2608.27278cs.DCcs.CV2026-08

分离I/O与计算,实现大规模病理切片嵌入高效提取

Decoupled I/O-Dominant Pipelines for Large-Scale Whole-Slide Image Embedding Extraction

论文配图:Decoupled I/O-Dominant Pipelines for Large-Scale Whole-Slide Image Embedding Extraction
图 1 · 摘自论文原文
  • 将切片处理分为三阶段:打块、并行推理、分片入库
  • 支持多节点扩展,存储开销在高并发下成主要瓶颈
  • 生成带元数据的可复用嵌入库,适合低资源场景

全切片图像(WSIs)是计算病理学的核心,但体积过大,实际中通常以补丁为单位进行基础模型推理。在大规模场景下,生成和处理海量补丁会引入显著的I/O与调度开销,常成为端到端性能瓶颈。本文提出一种解耦的、面向I/O的流水线,将流程分解为三个阶段:(1) 补丁生成与暂存,(2) 可并行的嵌入推理,(3) 分片向量数据库写入。该设计将数据移动与计算隔离,实现高效补丁供给和低通信多节点推理。系统输出一个分布式向量数据库,嵌入与患者、切片、补丁等丰富元数据持久关联,支持高效过滤、检索与下游复用。该表示数据库紧凑且可重复使用于检索、分类及少样本学习,尤其适用于低资源环境。通过刻画扩展边界,我们发现当并发度超过一定水平后,存储主导整体性能,将WSI嵌入提取重新定义为以数据为中心的系统问题,而非单纯计算密集型任务。

原文摘要 · Abstract (English)

Whole-slide images (WSIs) are central to computational pathology but are prohibitively large, making patch-based processing the practical unit for foundation model inference. At scale, however, generating and handling massive numbers of patches on quickly introduces significant I/O and orchestration overhead, often dominating end-to-end performance. We present a decoupled, I/O-aware pipeline for large-scale WSI embedding extraction that decomposes the workflow into three stages: (1) patch generation and staging, (2) embarrassingly parallel embedding inference, and (3) sharded vector database ingestion. This design isolates data movement from compute, enabling efficient patch delivery, scalable multi-node inference with minimal communication. The resulting system produces a distributed vector database where embeddings are persistently coupled with rich metadata (e.g., patient, slide, and patch attributes), enabling efficient filtering, retrieval, and downstream reuse. This representation database is compact and reusable for tasks such as retrieval, classification, and few-shot learning, particularly benefiting low-resource environments. We show that decoupling I/O, computation, and ingestion enables high-throughput WSI embedding extraction at scale. By characterizing the scaling envelope, we demonstrate that storage dominates beyond moderate concurrency, reframing WSI embedding extraction as a data-centric systems problem rather than a purely compute-bound workload.

病理图像嵌入提取系统优化向量数据库

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。