arXiv:2502.17378cs.SEcs.LG2025-02被引 1

ML项目持续集成难,因模型训练和数据依赖导致构建慢、测试难。

Continuous Integration Practices in Machine Learning Projects: The Practitioners` Perspective

  • 通过调研155名从业者,分析ML项目CI差异根源。
  • 构建时长更长、测试覆盖率更低,受数据与计算资源制约。
  • 建议追踪模型指标、优化测试优先级,加强跨团队协作。

持续集成(CI)是现代软件开发的核心实践。尽管在传统软件中广泛应用,但将CI应用于机器学习(ML)项目时面临独特挑战。此前研究发现,ML项目构建时间更长、测试覆盖率更低。本研究基于此,对47个ML项目的155名从业者进行调查,从定性角度探究这些差异的原因。从业者指出八大关键差异,包括测试复杂性、基础设施需求、构建时长与稳定性等。主要挑战包括项目复杂度高、模型训练负担重、数据处理量大、计算资源需求高及依赖管理困难,均导致构建时间延长。此外,ML系统的非确定性、数据依赖性和计算约束被识别为有效测试的主要障碍。研究结论表明,虽基础CI原则仍适用,但需针对ML特性定制方案。为此,提出一系列面向ML的CI实践,如跟踪模型性能指标、在CI流水线中优先执行关键测试。同时强调跨学科协作对强化测试文化的重要性。通过结合量化数据与从业者洞察,本研究深化了对CI与ML项目特殊需求之间相互作用的理解,为构建更高效、稳健的ML CI策略奠定基础。

原文摘要 · Abstract (English)

Continuous Integration (CI) is a cornerstone of modern software development. However, while widely adopted in traditional software projects, applying CI practices to Machine Learning (ML) projects presents distinctive characteristics. For example, our previous work revealed that ML projects often experience longer build durations and lower test coverage rates compared to their non-ML counterparts. Building on these quantitative findings, this study surveys 155 practitioners from 47 ML projects to investigate the underlying reasons for these distinctive characteristics through a qualitative perspective. Practitioners highlighted eight key differences, including test complexity, infrastructure requirements, and build duration and stability. Common challenges mentioned by practitioners include higher project complexity, model training demands, extensive data handling, increased computational resource needs, and dependency management, all contributing to extended build durations. Furthermore, ML systems' non-deterministic nature, data dependencies, and computational constraints were identified as significant barriers to effective testing. The key takeaway from this study is that while foundational CI principles remain valuable, ML projects require tailored approaches to address their unique challenges. To bridge this gap, we propose a set of ML-specific CI practices, including tracking model performance metrics and prioritizing test execution within CI pipelines. Additionally, our findings highlight the importance of fostering interdisciplinary collaboration to strengthen the testing culture in ML projects. By bridging quantitative findings with practitioners' insights, this study provides a deeper understanding of the interplay between CI practices and the unique demands of ML projects, laying the groundwork for more efficient and robust CI strategies in this domain.

持续集成机器学习工程实践

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。