用纯CPU分析预测深度学习模型显存占用，避免训练时显存溢出。
Accurate GPU Memory Prediction for Deep Learning Jobs through Dynamic Analysis
- 通过纯CPU动态分析预测模型峰值显存，无需访问真实GPU。
- 相比基线方法，误差降低84%，显存溢出概率下降73%。
- 适合资源紧张环境下提前规划任务调度，提升集群效率。
深度学习对GPU资源带来巨大压力，尤其在GPU集群中，显存溢出（OOM）是阻碍模型训练和资源高效利用的主要障碍。传统OOM估算方法依赖静态图分析或直接GPU内存采样，前者难以捕捉模型动态，后者加剧了稀缺GPU资源的竞争。为此提出VeritasEst，一种完全基于CPU的分析工具，可在不接触目标GPU的情况下准确预测深度学习训练任务的峰值显存需求。其“离线”预测能力使系统能在任务调度前获取精确内存占用信息，有效防止OOM并优化GPU分配。在数千次针对卷积神经网络（CNN）模型的实验中验证：相比基线内存估计算法，VeritasEst将相对误差降低84%，显存估算失败概率减少73%。该工作为资源受限环境下的高效、可预测深度学习训练提供了关键进展。
原文摘要 · Abstract (English)
The benefits of Deep Learning (DL) impose significant pressure on GPU resources, particularly within GPU cluster, where Out-Of-Memory (OOM) errors present a primary impediment to model training and efficient resource utilization. Conventional OOM estimation techniques, relying either on static graph analysis or direct GPU memory profiling, suffer from inherent limitations: static analysis often fails to capture model dynamics, whereas GPU-based profiling intensifies contention for scarce GPU resources. To overcome these constraints, VeritasEst emerges. It is an innovative, entirely CPU-based analysis tool capable of accurately predicting the peak GPU memory required for DL training tasks without accessing the target GPU. This "offline" prediction capability is core advantage of VeritasEst, allowing accurate memory footprint information to be obtained before task scheduling, thereby effectively preventing OOM and optimizing GPU allocation. Its performance was validated through thousands of experimental runs across convolutional neural network (CNN) models: Compared to baseline GPU memory estimators, VeritasEst significantly reduces the relative error by 84% and lowers the estimation failure probability by 73%. VeritasEst represents a key step towards efficient and predictable DL training in resource-constrained environments.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。