arXiv:2512.18608cs.CL2025-12被引 1

轻量模型可有效实现隐私信息掩码,兼顾效率与数据安全。

A Comparative Study of Light-weight Language Models for PII Masking and their Deployment for Real Conversational Texts

  • 对比T5-small与Mistral-Instruct-v0.3在24类标准隐私信息上的表现
  • 两者性能接近前沿大模型,但Mistral召回率更高、生成延迟高
  • T5结构化输出可控、推理成本低,适合实时对话系统部署

自动化个人身份信息(PII)掩码对隐私保护对话系统至关重要。尽管前沿大模型表现出色,但数据处理与计算成本引发对其替代方案的探索。本文通过微调T5-small和Mistral-Instruct-v0.3,在基于AI4Privacy基准构建的英文数据集上比较编码器-解码器与仅解码器架构。数据集涵盖24种标准化PII类别及更细粒度设置。采用实体级与字符级指标、类型准确率与精确匹配评估发现,两类轻量模型在PII掩码任务中表现与前沿模型相当。标签规范化显著提升性能。Mistral在多数类型上实现更高F1与召回率,但生成延迟显著增加;而T5虽在对话文本中鲁棒性较弱,却具备更强的结构化输出控制力与更低推理成本,适用于实时Discord机器人部署。对真实消息的评估显示,非正式输入下性能下降。研究揭示了准确性、鲁棒性与计算效率间的权衡,证明轻量模型可在解决大模型数据隐患的同时提供高效有效的PII掩码。

原文摘要 · Abstract (English)

Automated masking of Personally Identifiable Information (PII) is critical for privacy-preserving conversational systems. While current frontier large language models demonstrate strong PII masking capabilities, concerns about data handling and computational costs motivate exploration of whether lightweight models can achieve comparable performance. We compare encoder-decoder and decoder-only architectures by fine-tuning T5-small and Mistral-Instruct-v0.3 on English datasets constructed from the AI4Privacy benchmark. We create different dataset variants to study label standardization and PII representation, covering 24 standardized PII categories and higher-granularity settings. Evaluation using entity-level and character-level metrics, type accuracy, and exact match shows that both lightweight models achieve performance comparable to frontier LLMs for PII masking tasks. Label normalization consistently improves performance across architectures. Mistral achieves higher F1 and recall with greater robustness across PII types but incurs significantly higher generation latency. T5, while less robust in conversational text, offers more controllable structured outputs and lower inference cost, motivating its use in a real-time Discord bot for real-world PII redaction. Evaluation on live messages reveals performance degradation under informal inputs. These results clarify trade-offs between accuracy, robustness, and computational efficiency, demonstrating that lightweight models can provide effective PII masking while addressing data handling concerns associated with frontier LLMs.

隐私保护轻量模型语言模型数据安全

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。