简单微调比复杂架构更有效,实现全面的敏感信息识别
Fine-Tuning Over Architectural Complexity: Broad-Coverage PII Detection on PIIBench with DeBERTa
- 直接微调DeBERTa模型,用加权交叉熵优化
- 在5000条测试集上达到0.6476的F1分数,领先其他系统
- 适合需要高覆盖、跨源敏感信息检测的落地场景
敏感信息(PII)检测系统常在狭窄来源或领域内训练,导致在异构文本上覆盖不足。本文研究在包含82种实体类型、来自十个数据集的多源修正版PIIBench上的模型微调。评估了三种基于DeBERTa的方法:直接标记分类微调、源条件分层模型(SC+H),以及三阶段课程扩展(SC+H+Curr)。在可复现的5000条保留子集(test_5k)上,直接微调的DeBERTa取得F1 0.6476,SC+H和课程变体分别为0.5899和0.2772;最强对比系统仅达0.1723。因验证初期偏好SC+H,我们对完整100,002条保留集进行流式评估,直接微调仍最优,F1为0.6455,优于SC+H的0.5894。实体级分析显示,直接微调在54/82个细粒度实体类型及全部10个粗粒度组中胜出,支持加权实体F1;SC+H仅在28个类型上有局部优势。结果表明,多样化的任务特定训练数据与简单加权交叉熵目标,比所测试的架构与课程复杂性更有利于广覆盖的PII检测。
原文摘要 · Abstract (English)
Personally identifiable information (PII) detection systems are frequently trained within narrow source or domain boundaries, limiting coverage when deployed on heterogeneous text. We study model fine-tuning on a corrected multi-source PIIBench preparation spanning 82 retained entity types across ten source datasets. We evaluate three DeBERTa-based approaches: direct token classification fine-tuning, a source-conditioned hierarchical model (SC+H), and a three-phase curriculum extension (SC+H+Curr). Against eight published comparator systems on a reproducible 5,000-record held-out subset (test_5k), direct fine-tuned DeBERTa achieves F1 0.6476, while SC+H and the curriculum variant achieve 0.5899 and 0.2772 respectively; the strongest published comparator reaches only 0.1723. Because validation initially favoured SC+H, we perform a final streamed evaluation on the complete 100,002-record held-out split. Direct fine-tuning remains superior, achieving F1 0.6455 versus 0.5894 for SC+H. Entity-level analysis shows that direct fine tuning wins 54 of 82 fine entity types and all ten coarse groups by support-weighted entity F1, while SC+H retains localised advantages on 28 types. The results indicate that diverse task-specific training data and a simple weighted cross-entropy objective contribute more to broad-coverage PII detection than the tested architectural and curriculum complexity.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。