通过复用相似数据分布的模型,显著降低机器学习运维的维护成本和时间。
An Efficient Model Maintenance Approach for MLOps
- 基于时间序列数据的周期性分布模式,实现模型复用。
- 相比基线方法,计算耗时与成本降低至1/8,性能相当。
- 适合需要长期维护模型的工业界实践者使用。
近年来,众多行业在系统中应用机器学习(ML)模型。理想情况下,模型应基于同分布的数据训练与应用,但实际中数据随时间演变,引发概念漂移,导致模型性能下降。因此,保持模型更新对MLOps流程至关重要。现有维护方法通常计算资源消耗大、成本高、耗时且依赖特定模型。为此,我们提出改进的MLOps流水线、新型模型维护方法及相似性驱动的模型复用工具SimReuse。初步研究发现时间序列数据存在季节性和周期性分布模式,这些模式使未来可复用先前训练好的模型。我们将模型复用机制集成至MLOps流水线,并开发SimReuse工具,用于存储模型并在未来对相似数据分布的片段进行推理时复用。在五个时间序列数据集上的评估表明,该方法在维持模型性能的同时,大幅减少维护时间、成本与重训次数。相比最优基线,本方法性能相当,但计算时间与成本降至1/8。工业界与从业者可借此提升部署阶段的模型维护效率,降低时间和成本。
原文摘要 · Abstract (English)
In recent years, many industries have utilized machine learning (ML) models in their systems. Ideally, ML models should be trained on and applied to data from the same distributions. However, the data evolves over time in many application areas, leading to concept drift, which in turn causes the performance of the ML models to degrade over time. Therefore, maintaining up-to-date ML models plays a critical role in the MLOps pipeline. Existing ML model maintenance approaches are often computationally resource-intensive, costly, time-consuming, and model-dependent. Thus, we propose an improved MLOps pipeline, a new model maintenance approach and a Similarity-Based Model Reuse (SimReuse) tool to address the challenges of ML model maintenance. We identify seasonal and recurrent data distribution patterns in time series datasets throughout a preliminary study. Recurrent data distribution patterns enable us to reuse previously trained models for similar distributions in the future, thus avoiding frequent unnecessary retrainings. Then, we integrated the model reuse approach into the MLOps pipeline and proposed our improved MLOps pipeline. Furthermore, we develop SimReuse, a tool to implement the new components of our MLOps pipeline to store models and reuse them for inference of data segments with similar data distributions in the future. Our evaluation results on five time series datasets demonstrate that our model reuse approach can maintain the models' performance while significantly reducing maintenance time, costs, and the number of retrainings. Our model reuse approach achieves ML model performance comparable to the best baselines, while reducing the computation time and costs to 1/8th. Therefore, industries and practitioners can benefit from our approach and use our tool to maintain their ML models' performance in the deployment phase to reduce their maintenance time and costs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。