统一缓存框架IGTCache提升异构AI任务的缓存效率
Efficient Unified Caching for Accelerating Heterogeneous AI Workloads
- 用分层访问树结构分析数据访问模式
- 缓存命中率提升55.6%，任务完成时间减少52.2%
- 适合多类型AI工作负载共用缓存的场景
现代AI集群运行数据预处理、训练和推理等多样化任务，通常将大量数据存储在云存储中，并依赖缓存框架加速远程访问。为避免代码侵入和减少缓存空间浪费，理想方案是所有任务共享一个统一缓存。然而，现有缓存策略针对特定任务设计，难以应对集群中异构的访问模式和数据粒度。本文提出IGTCache，一种面向现代AI集群的统一高效缓存系统。IGTCache采用分层访问抽象AccessStreamTree，将近期数据访问组织为树结构，支持在不同粒度下检测访问模式。基于该结构，IGTCache通过假设检验将访问模式分类为顺序、随机或倾斜。根据识别出的模式与粒度，动态调整预取、淘汰和空间分配策略。实验表明，IGTCache相较当前最优缓存框架，缓存命中率提升55.6%，整体作业完成时间减少52.2%。
原文摘要 · Abstract (English)
Modern AI clusters, which host diverse workloads like data pre-processing, training and inference, often store the large-volume data in cloud storage and employ caching frameworks to facilitate remote data access. To avoid code-intrusion complexity and minimize cache space wastage, it is desirable to maintain a unified cache shared by all the workloads. However, existing cache management strategies, designed for specific workloads, struggle to handle the heterogeneous AI workloads in a cluster -- which usually exhibit heterogeneous access patterns and item storage granularities. In this paper, we propose IGTCache, a unified, high-efficacy cache for modern AI clusters. IGTCache leverages a hierarchical access abstraction, AccessStreamTree, to organize the recent data accesses in a tree structure, facilitating access pattern detection at various granularities. Using this abstraction, IGTCache applies hypothesis testing to categorize data access patterns as sequential, random, or skewed. Based on these detected access patterns and granularities, IGTCache tailors optimal cache management strategies including prefetching, eviction, and space allocation accordingly. Experimental results show that IGTCache increases the cache hit ratio by 55.6% over state-of-the-art caching frameworks, reducing the overall job completion time by 52.2%.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。