arXiv:2609.03464cs.LG2026-09

实测发现主流隐私信息检测系统在真实场景下鲁棒性差,易漏检。

Mind the Gap: Robustness Risks in PII Detection Systems

论文配图:Mind the Gap: Robustness Risks in PII Detection Systems
图 1 · 摘自论文原文
  • 构建七类自然分布偏移的应力测试集,评估三类主流模型表现
  • 三类模型均在分布外数据上性能显著下降,失败模式各不相同
  • 提出带问答反馈的混合检测流水线,助力实际部署风险控制

个人身份信息(PII)检测是数据保护基础设施的核心组件,漏检直接带来隐私与安全风险。尽管现代PII系统在标准基准上表现优异,但我们发现其评估结果掩盖了在实际部署中遭遇分布偏移时的严重鲁棒性缺陷。不同于仅比较准确率,本文研究不同检测范式在噪声、非结构化及非正式输入下的失效情况。构建涵盖七类自然分布偏移的应力测试基准,评估三种广泛部署架构:基于编码器的命名实体识别(SpaCy)、基于规则的混合检测(Presidio)以及生成式大模型抽取(Qwen2.5-3B)。所有三类系统在分布外输入上均出现显著性能下降,且失败模式各异:编码器模型主要在未见表面形式和边界定位上出错,规则系统在非标准格式下失效,大模型则表现出实体类型混淆与生成不稳定。结果表明,整体基准得分无法反映部署关键弱点,单一架构无法在各类PII中保持一致可靠性。基于此,我们提出一种基于问答反馈的混合检测流水线,用于迭代风险缓解,并发布该基准以支持分布外感知的PII系统评估。

原文摘要 · Abstract (English)

Personally Identifiable Information (PII) detection is a foundational component of data protection infrastructure where missed entities constitute direct privacy and security risks. Although modern PII systems report strong performance on standard benchmarks, we show that these evaluations mask substantial robustness failures under realistic distribution shifts encountered in deployment. Rather than comparing state-of-the-art accuracy, we study how different PII detection paradigms fail under noisy, unstructured, and informal inputs. We construct a stress test benchmark spanning seven categories of natural distribution shift and evaluate representative systems from three widely deployed architectural families: encoder-based NER (SpaCy), rule-based hybrid detection (Presidio), and generative LLM extraction (Qwen2.5-3B). All three exhibit significant degradation on out-of-distribution inputs, but with distinct and complementary failure modes. Encoder models primarily fail on unseen surface forms and boundary detection, rule-based systems fail on non-standard formats, and LLMs exhibit entity-type confusion and generation instability. These results show that aggregate benchmark scores obscure deployment-critical weaknesses and that no single architecture is uniformly reliable across PII categories. Motivated by these findings, we propose a hybrid detection pipeline with a QA-driven feedback loop for iterative risk mitigation, and release our benchmark to support OOD-aware evaluation of PII systems.

PII检测鲁棒性大模型应用

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。