让领域专家不用懂技术也能自己检查数据质量
Domain-Specific Data Quality Analysis Using Technology-Independent Query Templates

- 用可复用的模板定义数据质量分析,不依赖具体数据库技术
- 在文化遗迹领域测试中,模板能发现真实数据问题
- 领域专家无需技术人员帮助即可独立完成质量检测
在数据驱动的时代,数据质量至关重要。由于质量标准具有领域和场景特异性,通常由领域专家定义质量要求,但他们往往缺乏查询语言技能,难以自行实施质量分析。这导致流程高度依赖技术人员,限制了领域专家的自主性。为此,我们提出质量模式模型框架(QPM),一种与数据库技术无关的模板化方法,用于定义数据质量分析。该方法可避免重复为不同数据库系统编写分析逻辑。我们针对XML、RDF和Neo4j三种数据库技术实现原型,并在文化遗迹领域开展定性用户研究,收集实际数据问题。结果表明,QPM在表达能力上可媲美甚至超过常见数据库查询语言,且使领域专家能在无技术支援下独立定义基于模板的质量分析。
原文摘要 · Abstract (English)
In an increasingly data-driven world, effectively working with data depends heavily on its quality. Quality analysis is a central aspect of data quality management. As data quality is typically domain- and context-specific, the definition of quality requirements is primarily the responsibility of domain experts. However, domain experts often lack the query language expertise needed to implement quality analyses. Therefore, the process of defining quality analyses results in a resource-intensive workflow that requires the involvement of technical experts, effectively excluding domain experts from independently managing data quality. To address this challenge, we present the Quality Pattern Model framework (QPM), a model-driven approach to define templates for data quality analyses that are independent of specific database technologies and application domains. QPM can eliminate the need for deep technical expertise and prevent the need for defining quality analyses several times for different database technologies. We present a proof-of-concept implementation of this approach for three database technologies: XML, RDF, and Neo4j. We evaluate the expressiveness of our approach, its applicability in the cultural heritage domain, and its usability by domain experts. For this purpose, we conducted a qualitative user study and empirically collected quality problems in a catalog. Our findings suggest that QPM matches and even exceeds the expressiveness of common database query languages. Furthermore, the results indicate that our tool enables domain experts to define template-based quality analyses independently, without requiring support of IT experts.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。