梳理数据质量意识从传统系统到智能时代的演进路径
Data Quality Awareness: A Journey from Traditional Data Management to Data Science Systems
- 从传统数据管理到数据科学，分析数据质量挑战的演变
- 揭示机器学习与大数据质量问题间的因果关联
- 为数据科学家提供系统性质量认知框架
人工智能已深刻改变多个领域并影响日常生活，其成功关键在于高质量数据。本文全面回顾了数据质量（DQ）意识从传统数据管理系统向现代数据驱动的AI系统演进的过程，涵盖大数据与机器学习领域的质量挑战及应对技术。聚焦于由机器学习驱动的分析活动，本文通过揭示机器学习与大数据质量问题之间的因果联系，深化对数据科学系统中新兴数据质量挑战及其感知技术的理解。据我们所知，本文是首个系统梳理传统与新兴数据科学系统中数据质量意识演进的综述。希望读者能从中获得洞见与价值。
原文摘要 · Abstract (English)
Artificial intelligence (AI) has transformed various fields, significantly impacting our daily lives. A major factor in AI success is high-quality data. In this paper, we present a comprehensive review of the evolution of data quality (DQ) awareness from traditional data management systems to modern data-driven AI systems, which are integral to data science. We synthesize the existing literature, highlighting the quality challenges and techniques that have evolved from traditional data management to data science including big data and ML fields. As data science systems support a wide range of activities, our focus in this paper lies specifically in the analytics aspect driven by machine learning. We use the cause-effect connection between the quality challenges of ML and those of big data to allow a more thorough understanding of emerging DQ challenges and the related quality awareness techniques in data science systems. To the best of our knowledge, our paper is the first to provide a review of DQ awareness spanning traditional and emergent data science systems. We hope that readers will find this journey through the evolution of data quality awareness insightful and valuable.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。