分析1993个开源机器学习项目容器实践，揭示构建效率瓶颈与优化模式。
ML in a Box: Analyzing Containerization Practices in Open Source ML Projects
- 统计1993个ML项目Dockerfile，发现容器角色分训练、推理和基础设施三类
- 容器平均10.27GB大，构建需8.84分钟，44.4%提交触发重建
- 提炼7种通用优化模式，可减少冗余计算，提升构建效率
容器化在机器学习领域日益重要，提供可复现性、可移植性和环境一致性。尽管已有研究分析Dockerfile结构与最佳实践，但尚未深入探讨机器学习工作流的迭代特性如何影响容器体积、构建性能和缓存行为。本研究首次对1,993个相关Dockerfile开展大规模实证分析，结合定量统计容器在项目中的角色与构建动态，以及定性考察重构实践。结果表明，容器在训练、推理和基础设施中承担不同角色；容器普遍较大，平均大小为10.27 GB，构建时间约8.84分钟。44.4%的提交引发重建，主要因上下文文件变更（96.4%），且实验是触发重建的主要动机。尽管部分缓存可复用，仍有71%的重建工作浪费在冗余计算上。从稳定项目中识别出7种常见的机器学习专用Dockerfile重构模式，能有效提升构建效率并减小容器体积。
原文摘要 · Abstract (English)
Containerization has become increasingly essential in the machine learning (ML) domain, providing reproducibility, portability, and environment consistency. While prior studies have analyzed Dockerfile structures and best practices, none have examined ML projects in depth to reveal how the iterative nature of ML workflows influences container footprint, build performance, and caching behavior. We present the first large scale empirical study of 1,993 ML related Dockerfiles, combining quantitative analysis of container roles in ML projects and build dynamics with a qualitative investigation of refactoring practices. Results show that containers serve distinct roles across training, inference, and infrastructure. Containers are typically large, averaging 10.27 GB in size, and require long build times of about 8.84 minutes. We find that 44.4% of commits trigger rebuilds, primarily due to context file changes (96.4%), with experimentation being the main motive behind those commits that initiate rebuilds. Despite partial cache reuse, 71% of rebuild work is wasted on redundant computation. From stable projects, we identify 7 recurring ML-specific Dockerfile refactoring patterns that improve build efficiency and reduce container footprint.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。