让领域专家零代码定义数据质量规则,自动转为可执行检查。
A Model-Driven Pipeline for Data Quality Specification and Operationalization: A No-Code Approach for Domain Experts

- 用自然语言描述规则,通过模板化工具链自动转为可执行分析
- 基于元模型构建可复用的质量检查模板,支持重复使用
- 适合无技术背景的领域专家快速提升数据质量
高质量数据对跨领域的可靠分析、决策与研究至关重要。在文化遗产等人工采集数据的领域,数据易出现不一致等质量问题。需通过系统化的质量分析验证数据是否符合领域预期,而这些预期由领域专家最清楚,他们可用自然语言表达。但专家通常缺乏将这些要求形式化为可执行分析的技术能力,导致必须依赖数据工程师协作,过程耗时且技术门槛高。为此,我们提出一个模型驱动的数据质量规范与落地流程。采用QPM元模型定义可复用的质量分析模板,通过名为Constrainify的Web应用,使专家能根据具体需求定制模板,并借助基于模型驱动工程的工具链转化为可执行分析。最终生成一系列可重复、可复用、领域特定的数据质量检查方案。
原文摘要 · Abstract (English)
High-quality data is essential for reliable analysis, decision-making, and research across domains. This is especially relevant in areas such as cultural heritage, where data is collected and curated manually, making it prone to quality issues like inconsistencies. To improve data quality, the data must be analyzed regularly using systematic quality analyses. Quality analyses validate the conformance of data to domain-specific expectations. These expectations are best understood by domain experts, who can express them using natural language. However, they rarely possess the technical expertise to formalize these expectations into executable quality analyses. Consequently, this process requires domain experts and data engineers, making it time-consuming and technically demanding. The required technical expertise and the resulting dependencies pose a significant challenge. To address this challenge, we present a pipeline for formalizing and operationalizing data quality constraints. We support this pipeline using QPM, a metamodel for defining templates for reusable quality analyses. The web application Constrainify enables tailoring templates to specific conceptual requirements and translating them into executable quality analyses via a tool-chain based on model-driven engineering subpipelines. The result is a set of reusable, repeatable, and domain-specific quality analyses.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。