将医疗语音诊断的机器学习流水线从原型升级为可生产系统
Making a Pipeline Production-Ready: Challenges and Lessons Learned in the Healthcare Domain
- 分三阶段重构：从混乱代码到模块化单体，再到测试驱动微服务
- 每轮迭代显著提升系统可扩展性、可维护性和容错能力
- 适合想把科研模型落地到实际系统的开发者和研究者
将机器学习训练流水线投入生产需具备良好的工程实践。然而，典型的数据科学流程常导致代码缺乏关键软件质量属性。本文以SPIRA项目为例，该项目旨在通过语音分析实现呼吸功能不足的早期诊断，构建一个机器学习增强系统（MLES）。论文概述了MLES架构，并对比了其持续训练子系统的三个版本：初始概念验证阶段的‘大泥球’（v1）、基于设计模式的模块化单体（v2），以及测试驱动的微服务集合（v3）。每一轮演进均显著提升了系统的可扩展性、可维护性、鲁棒性和弹性。文章总结了该过程中的挑战与经验教训，为希望将模型流水线投入生产的科研人员和从业者提供实用洞见。
原文摘要 · Abstract (English)
Deploying a Machine Learning (ML) training pipeline into production requires good software engineering practices. Unfortunately, the typical data science workflow often leads to code that lacks critical software quality attributes. This experience report investigates this problem in SPIRA, a project whose goal is to create an ML-Enabled System (MLES) to pre-diagnose insufficiency respiratory via speech analysis. This paper presents an overview of the architecture of the MLES, then compares three versions of its Continuous Training subsystem: from a proof of concept Big Ball of Mud (v1), to a design pattern-based Modular Monolith (v2), to a test-driven set of Microservices (v3) Each version improved its overall extensibility, maintainability, robustness, and resiliency. The paper shares challenges and lessons learned in this process, offering insights for researchers and practitioners seeking to productionize their pipelines.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。