用大模型生成真实表格错误,提升数据清洗评测可靠性
Towards Practical Benchmarking of Data Cleaning Techniques: On Generating Authentic Errors via Large Language Models
- 基于表格微调的LLM框架,建模错误生成-检测-修正全流程
- 生成错误与真实数据分布相似度超规则方法和未微调的LLM
- 适合数据清洗算法评测,尤其对机器学习类检测方法有效
数据质量是数据驱动系统中的关键挑战,表格数据中的错误会严重损害下游分析与机器学习性能。尽管已有众多错误检测算法,但缺乏多样且真实的错误数据集限制了全面评估。人工标注耗时且不一致,促使人们探索合成错误生成。本文提出TableEG框架,利用大语言模型(LLMs)生成真实可信的错误。通过表格微调策略与三元组表示$(I, T, O)$建模错误生成、检测与纠正任务,表征二维表格中复杂的依赖关系。在12个涵盖10个不同领域的真实数据集上训练,确保合成错误忠实反映真实错误分布。实验表明,TableEG生成的错误在模式与分布上均优于规则方法及未微调的LLM生成结果。在几乎所有数据集与检测算法下,其性能指标与真实错误表现高度一致,尤其对基于机器学习的检测技术效果显著。总体而言,TableEG不仅弥合了合成与真实错误之间的差距,还为后续错误检测与修复任务建立了可靠基准。
原文摘要 · Abstract (English)
Data quality remains an important challenge in data-driven systems, as errors in tabular data can severely compromise downstream analytics and machine learning performance. Although numerous error detection algorithms have been proposed, the lack of diverse, real-world error datasets limits comprehensive evaluation. Manual error annotation is both time-consuming and inconsistent, motivating the exploration of synthetic error generation as an alternative. In this work, we introduce TableEG, a framework that leverages large language models (LLMs) to generate authentic errors. By employing a table fine-tuning strategy and a triplet representation $(I, T, O)$ to model error generation, detection, and correction tasks, TableEG captures the complex dependencies inherent in two-dimensional tables. Trained on 12 real-world datasets spanning 10 diverse domains, TableEG ensures that the synthesized errors faithfully reflect authentic error distributions. Experimental results indicate that errors generated by TableEG exhibit superior pattern and distribution similarity compared to both rule-based methods and LLM-generated errors without fine-tuning. Furthermore, performance metrics on TableEG-generated errors closely align with those on real-world errors across nearly all datasets and detection algorithms, particularly for machine learning based detection techniques. Overall, TableEG not only bridges the gap between synthetic and real-world errors but also establishes a robust benchmark for subsequent error detection and correction tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。