用大模型实现无需预设标签体系的隐私数据检测
Addressing Labelled Data Scarcity: Taxonomy-Agnostic Annotation of PII Values in HTTP Traffic using LLMs
- 构建多阶段大模型流程,动态解析任意标签体系下的隐私信息
- 在3个不同领域税则下准确识别并提取隐私值,召回率超90%
- 自动生成带验证标注的合成流量数据,适合隐私审计与模型训练
自动化隐私审计常通过分析出站HTTP流量来检测个人身份信息(PII)泄露。现有基于学习的检测器通常依赖稀缺的手动标注流量且紧耦合于固定标签体系,限制了跨领域迁移和对演变中PII定义的适应性。本文研究大语言模型(LLMs)能否在运行时提供标签体系的情况下,支持对HTTP消息体中显式传输的PII值进行无分类体系依赖的标注。提出一种多阶段基于LLM的流水线,结合确定性预处理、标签级分类、目标实例级值标注及输出验证。为支持可控评估与示例提示而不依赖真实用户数据,还设计了一种基于LLM的合成HTTP流量生成器,其带有经人工验证的、源自税则的PII标注。在三个涵盖不同PII领域与粒度层级的税则上评估该方法。结果表明,该流水线能准确检测PII类型并提取对应值。总体而言,研究结果表明,大模型可作为灵活、无分类体系依赖的流量标注基础,并支持在不断演进的隐私税则下生成标注数据。
原文摘要 · Abstract (English)
Automated privacy audits of web and mobile applications often analyse outbound HTTP traffic to detect Personally Identifiable Information (PII) leakage. However, existing learning-based detectors typically depend on scarce, manually labelled traffic and are tightly coupled to fixed label taxonomies, limiting transferability across domains and evolving definitions of PII. This paper investigates whether Large Language Models (LLMs) can support taxonomy-agnostic annotation of explicitly transmitted PII values in HTTP message bodies when the taxonomy is provided at runtime. We introduce a multi-stage LLM-based pipeline that combines deterministic pre-processing with label-level classification, targeted instance-level value annotation, and output validation. To enable controlled evaluation and exemplar-based prompting without relying on sensitive real-user captures, we further propose an LLM-based generator for synthetic HTTP traffic with manually validated, taxonomy-derived PII annotations. We evaluate the approach across three taxonomies spanning different PII domains and granularity levels. Results show that the pipeline accurately detects PII types and extracts corresponding values for concrete PII taxonomies. Overall, our findings position LLMs as a promising foundation for flexible, taxonomy-agnostic traffic annotation and for creating labelled data under evolving privacy taxonomies.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。