对比五种模型格式,发现ONNX在系统集成中最高效易用。
How Do Model Export Formats Impact the Development of ML-Enabled Systems? A Case Study on Model Integration
- 通过30个案例对比ONNX、SavedModel等格式的集成效果。
- ONNX在跨平台移植和集成效率上表现最佳,Pickle最难使用。
- 适合需要跨框架部署或复杂预处理的开发者参考选择。
机器学习模型常被集成到ML系统中以实现难以通过传统编程达成的功能。这一过程需选择合适的模型导出格式,现有选项众多。但当前缺乏实证依据指导实践者进行格式选择。为此,我们基于17人的初步问卷调查,设计了一个包含两个ML系统、三种技术栈的嵌入式案例研究,共分析30个实例(2系统×3技术栈×5格式),收集结构化现场笔记。结果表明:ONNX在多数情况下提供了最高效的集成与跨平台兼容性;SavedModel和TorchScript在纯Python系统中使用便捷,但在其他环境需额外适配(尤其TorchScript);SavedModel可将预处理逻辑整合至单一文件,利于复杂深度学习场景扩展;而Pickle与Joblib即便在Python环境中也最难集成。所有格式均具备完善的文档与社区支持(如Stack Overflow、Reddit)。本研究为开发者选择适配自身场景的模型导出格式提供了实证依据。
原文摘要 · Abstract (English)
Machine learning (ML) models are often integrated into ML-enabled systems to provide software functionality that would otherwise be impossible. This integration requires the selection of an appropriate ML model export format, for which many options are available. These formats are crucial for ensuring a seamless integration, and choosing a suboptimal one can negatively impact system development. However, little evidence is available to guide practitioners during the export format selection. We therefore evaluated various model export formats regarding their impact on the development of ML-enabled systems from an integration perspective. Based on the results of a preliminary questionnaire survey (n=17), we designed an extensive embedded case study with two ML-enabled systems in three versions with different technologies. We then analyzed the effect of five popular export formats, namely ONNX, Pickle, TensorFlow's SavedModel, PyTorch's TorchScript, and Joblib. In total, we studied 30 units of analysis (2 systems x 3 tech stacks x 5 formats) and collected data via structured field notes. The holistic qualitative analysis of the results indicated that ONNX offered the most efficient integration and portability across most cases. SavedModel and TorchScript were very convenient to use in Python-based systems, but otherwise required workarounds (TorchScript more than SavedModel). SavedModel also allowed the easy incorporation of preprocessing logic into a single file, which made it scalable for complex deep learning use cases. Pickle and Joblib were the most challenging to integrate, even in Python-based systems. Regarding technical support, all model export formats had strong technical documentation and strong community support across platforms such as Stack Overflow and Reddit. Practitioners can use our findings to inform the selection of ML export formats suited to their context.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。