提出DataPro框架,提升数据科学项目落地效率
DataPro -- A Standardized Data Understanding and Processing Procedure: A Case Study of an Eco-Driving Project
- 在CRISP-DM基础上增加技术理解与实施阶段
- 案例显示可有效降低丹麦公交燃油消耗
- 适合需跨团队协作的数据科学项目
构建系统化数据处理与知识发现流程对从大数据中提取价值至关重要。尽管CRISP-DM是当前数据挖掘项目的事实标准,但数据处理技术的进步要求对其改进。本文提出DataPro(标准化数据理解与处理流程)模型,扩展了CRISP-DM,并通过增加“技术理解”和“实施”阶段强化数据科学家与利益相关者之间的衔接。其中,“技术理解”阶段确保技术团队准确理解业务目标;“实施”阶段聚焦模型在实际业务中的应用。明确各阶段职责有助于提升管理与沟通效率。以丹麦公共交通运输领域的节能驾驶数据科学项目为例,应用该框架后,项目明确了关键业务目标,将其转化为技术需求,并开发出可指导减排的模型。定性评估表明,该模型优于其他数据科学流程。
原文摘要 · Abstract (English)
A systematic pipeline for data processing and knowledge discovery is essential to extracting knowledge from big data and making recommendations for operational decision-making. The CRISP-DM model is the de-facto standard for developing data-mining projects in practice. However, advancements in data processing technologies require enhancements to this framework. This paper presents the DataPro (a standardized data understanding and processing procedure) model, which extends CRISP-DM and emphasizes the link between data scientists and stakeholders by adding the "technical understanding" and "implementation" phases. Firstly, the "technical understanding" phase aligns business demands with technical requirements, ensuring the technical team's accurate comprehension of business goals. Next, the "implementation" phase focuses on the practical application of developed data science models, ensuring theoretical models are effectively applied in business contexts. Furthermore, clearly defining roles and responsibilities in each phase enhances management and communication among all participants. Afterward, a case study on an eco-driving data science project for fuel efficiency analysis in the Danish public transportation sector illustrates the application of the DataPro model. By following the proposed framework, the project identified key business objectives, translated them into technical requirements, and developed models that provided actionable insights for reducing fuel consumption. Finally, the model is evaluated qualitatively, demonstrating its superiority over other data science procedures.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。