预测多模态模型训练时的显存峰值，避免训练中断
GPU Memory Prediction for Multimodal Model Training
- 分解模型层并因子化分析，估算每层显存占用
- 平均绝对百分比误差仅8.7%，预测精度高
- 适合大模型训练调度与资源规划人员
随着智能体系统中深度学习模型规模与复杂度不断增长，GPU显存需求持续上升，常超出可用容量，导致显存不足（OoM）错误。这类错误会中断整个训练过程，并浪费大量计算资源。因此，准确预测GPU显存使用量至关重要。然而，以往研究仅关注单模态架构，难以推广至多模态模型——而后者在智能体系统中十分常见。为此，本文提出一种框架，通过分析多模态模型的架构与训练行为，预测其峰值显存使用量。该框架将多模态模型分解为各组成层，并应用因子化方法估算每层的显存占用。评估结果显示，该框架在多组实验中实现约8.7%的平均绝对百分比误差（MAPE），表现出高预测精度。
原文摘要 · Abstract (English)
As deep learning models in agentic AI systems grow in scale and complexity, GPU memory requirements increase and often exceed the available GPU memory capacity, so that out-of-memory (OoM) errors occur. It is well known that OoM interrupts the whole training itself and wastes substantial computational resources. Therefore, to prevent OoM, accurate prediction of GPU memory usage is essential. However, previous studies focus only on unimodal architectures and fail to generalize to multimodal models, even though the multimodal models are a common choice in agentic AI systems. To address this limitation, we propose a framework that predicts the peak GPU memory usage by analyzing the model architecture and training behavior of multimodal models. Specifically, the framework decomposes the multimodal model into its constituent layers and applies factorization to estimate the memory usage of each layer. Our evaluation shows that our framework achieves high prediction accuracy of ~8.7% average MAPE.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。