提出混合模式检测算法,提升敏感信息识别的速度与准确率。
Decoding Complexity: Intelligent Pattern Exploration with CHPDA (Context Aware Hybrid Pattern Detection Algorithm)
- 结合正则与精确匹配,动态选择最优检测路径。
- 混合模型达91.6% F1分数,召回率高且误报少。
- 适配高低端设备,适合数据安全平台落地应用。
检测个人身份信息(PII)和受保护健康信息(PHI)对数据安全平台至关重要。本研究评估了基于正则表达式的匹配算法与精确匹配技术,以优化检测速度、准确率与可扩展性。基准测试显示,Google RE2在速度(10-15毫秒/MB)、内存效率(8-16兆字节)和准确率(99.5%)之间表现最佳,优于PCRE,且硬件兼容性高于Hyperscan。对于精确匹配,Aho-Corasick在大数据集上表现优异(8毫秒/MB)。性能分析表明,正则处理时间随数据量和模式复杂度线性增长。混合AI+正则方法通过提升召回率并减少误报,达到最高F1分数(91.6%)。设备测试证实该方案在高性能与中端系统上均保持高效CPU与内存使用。尽管有效,仍面临多语言支持有限与需持续更新模式等挑战。未来工作应拓展语言覆盖,融合数据安全与隐私管理(DSPM)与数据防泄漏(DLP)工具,并增强合规性以推动全球应用。
原文摘要 · Abstract (English)
Detecting sensitive data such as Personally Identifiable Information (PII) and Protected Health Information (PHI) is critical for data security platforms. This study evaluates regex-based pattern matching algorithms and exact-match search techniques to optimize detection speed, accuracy, and scalability. Our benchmarking results indicate that Google RE2 provides the best balance of speed (10-15 ms/MB), memory efficiency (8-16 MB), and accuracy (99.5%) among regex engines, outperforming PCRE while maintaining broader hardware compatibility than Hyperscan. For exact matching, Aho-Corasick demonstrated superior performance (8 ms/MB) and scalability for large datasets. Performance analysis revealed that regex processing time scales linearly with dataset size and pattern complexity. A hybrid AI + Regex approach achieved the highest F1 score (91. 6%) by improving recall and minimizing false positives. Device benchmarking confirmed that our solution maintains efficient CPU and memory usage on both high-performance and mid-range systems. Despite its effectiveness, challenges remain, such as limited multilingual support and the need for regular pattern updates. Future work should focus on expanding language coverage, integrating data security and privacy management (DSPM) with data loss prevention (DLP) tools, and enhancing regulatory compliance for broader global adoption.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。