用大模型检测结构化数据中的个人隐私,上下文信息让效果显著提升。
Detection of Personal Data in Structured Datasets Using a Large Language Model
- 利用大模型结合字段名、值及上下文信息进行检测。
- 在医疗数据上表现最佳,优于微软和CASSED工具。
- 真实数据集验证有效,适合隐私保护研究者使用。
我们提出一种新方法,利用GPT-4o检测结构化数据中的个人数据。关键创新在于引入上下文信息:不仅包含字段名和值,还融合了其他字段名及数据集描述。我们在多个数据集上对比该方法与Microsoft Presidio和CASSED的表现,包括DeSSI(大规模合成数据)、Kaggle和OpenML收集的数据,以及包含危重症患者信息的真实数据集MIMIC-Demo-Ext。结果表明,检测性能随数据集不同而显著变化:CASSED在训练所用的DeSSI上表现最佳;在医疗数据MIMIC-Demo-Ext上各模型表现相近,但本方法明显更优。在Kaggle和OpenML数据中,缺乏上下文信息的CASSED和Presidio表现较差,而基于上下文的GPT-4o方法效果突出。结论指出,该领域进展亟需更多真实世界含个人数据的公开数据集。
原文摘要 · Abstract (English)
We propose a novel approach for detecting personal data in structured datasets, leveraging GPT-4o, a state-of-the-art Large Language Model. A key innovation of our method is the incorporation of contextual information: in addition to a feature's name and values, we utilize information from other feature names within the dataset as well as the dataset description. We compare our approach to alternative methods, including Microsoft Presidio and CASSED, evaluating them on multiple datasets: DeSSI, a large synthetic dataset, datasets we collected from Kaggle and OpenML as well as MIMIC-Demo-Ext, a real-world dataset containing patient information from critical care units. Our findings reveal that detection performance varies significantly depending on the dataset used for evaluation. CASSED excels on DeSSI, the dataset on which it was trained. Performance on the medical dataset MIMIC-Demo-Ext is comparable across all models, with our GPT-4o-based approach clearly outperforming the others. Notably, personal data detection in the Kaggle and OpenML datasets appears to benefit from contextual information. This is evidenced by the poor performance of CASSED and Presidio (both of which do not utilize the context of the dataset) compared to the strong results of our GPT-4o-based approach. We conclude that further progress in this field would greatly benefit from the availability of more real-world datasets containing personal information.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。