用双向门控循环单元优化Transformer，更准预测深度学习的显存需求。
GPU Memory Requirement Prediction for Deep Learning Task Based on Bidirectional Gated Recurrent Unit Optimization Transformer
- 用BiGRU优化Transformer结构，提升内存预测模型的时序建模能力。
- 在MSE、RMSE、MAE和R2指标上均优于决策树等四种基准模型。
- 适合需要高效调度计算资源的研究者与系统工程师参考。
针对深度学习任务中对GPU内存资源需求日益增长的精准预测需求，本文深入分析了现有研究现状，创新性地提出一种融合双向门控循环单元（BiGRU）以优化Transformer架构的深度学习模型，旨在提升内存需求预测精度。为验证模型有效性，设计了对比实验，选取决策树、随机森林、Adaboost和XGBoost四种典型机器学习模型作为基准。实验结果表明，本文提出的BiGRU优化Transformer模型在关键评估指标上表现显著：在均方误差（MSE）和均方根误差（RMSE）方面，其值低于所有对比模型，预测结果与实际值偏差最小；在平均绝对误差（MAE）和决定系数（R²）指标上也表现良好，结果平衡且稳定，综合预测性能远超所比较的传统机器学习方法。综上，基于双向门控循环单元优化的Transformer模型成功构建了高效准确的深度学习任务显存需求预测框架，预测精度较传统机器学习方法有显著提升。该研究为优化深度学习任务的资源调度与管理提供了有力技术支持与可靠理论依据，有助于提升计算集群的利用效率。
原文摘要 · Abstract (English)
In response to the increasingly critical demand for accurate prediction of GPU memory resources in deep learning tasks, this paper deeply analyzes the current research status and innovatively proposes a deep learning model that integrates bidirectional gated recurrent units (BiGRU) to optimize the Transformer architecture, aiming to improve the accuracy of memory demand prediction. To verify the effectiveness of the model, a carefully designed comparative experiment was conducted, selecting four representative basic machine learning models: decision tree, random forest, Adaboost, and XGBoost as benchmarks. The detailed experimental results show that the BiGRU Transformer optimization model proposed in this paper exhibits significant advantages in key evaluation indicators: in terms of mean square error (MSE) and root mean square error (RMSE), the model achieves the lowest value among all comparison models, and its predicted results have the smallest deviation from the actual values; In terms of mean absolute error (MAE) and coefficient of determination (R2) indicators, the model also performs well and the results are balanced and stable, with comprehensive predictive performance far exceeding the benchmark machine learning methods compared. In summary, the Transformer model based on bidirectional gated recurrent unit optimization successfully constructed in this study can efficiently and accurately complete GPU memory demand prediction tasks in deep learning tasks, and its prediction accuracy has been significantly improved compared to traditional machine learning methods. This research provides strong technical support and reliable theoretical basis for optimizing resource scheduling and management of deep learning tasks, and improving the utilization efficiency of computing clusters.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。