跨36语言的隐私保护与实体识别框架，支持66亿人使用。
AWED-PIPER: Agents, Web Applications & Expert Detectors for Personally Identifiable Information Protection & Fine-grained Named Entity Recognition across 36 languages for 6.6 Billion Speakers
- 融合代理工具与脚本感知正则，实现多语言细粒度实体识别。
- 提供可逆伪匿名化，保留占位符与去匿名化映射关系。
- 覆盖低资源语言如博多语、曼尼普里语，适合隐私敏感场景。
命名实体识别（NER）和敏感个人信息（PII）匿名化是自然语言处理中信息提取与隐私保护的关键任务。我们提出AWED-PIPER，一个开源框架，包含代理工具、交互式网页应用及54个先进专家检测模型，支持跨36种语言（覆盖超66亿使用者）的细粒度命名实体识别（FgNER）与可逆合成型PII伪匿名化。系统结合细粒度多语言序列标注与脚本感知正则表达式，精准识别人物、地点、组织、医疗等上下文实体，以及邮箱、本地脚本手机号、IP地址、信用卡等结构化技术类PII。AWED-PIPER具备双重能力：完整实体抽取与隐私保护型可逆匿名化，通过持久占位符与去匿名化字典映射实现数据安全回溯。其资源覆盖全球语言至极低资源脆弱语言，如博多语、曼尼普里语、比什努普里亚语和米佐语。相关资源可通过以下链接获取：PII防护代理工具（https://github.com/PrachuryyaKaushik/AWED-PIPER）、FgNER代理工具（https://github.com/PrachuryyaKaushik/AWED-FiNER）、PII网页应用（https://hf.co/spaces/prachuryyaIITG/AWED_PII_Protector）、FgNER网页应用（https://hf.co/spaces/prachuryyaIITG/AWED-FiNER），以及边缘部署专家检测模型（https://hf.co/collections/prachuryyaIITG/awed-piper）。
原文摘要 · Abstract (English)
Named Entity Recognition (NER) and Personally Identifiable Information (PII) anonymization are critical tasks in Natural Language Processing (NLP) for information extraction and privacy preservation. We introduce AWED-PIPER, an open-source framework comprising agentic tools, interactive web applications, and 54 state-of-the-art expert detector models that provide unified Fine-grained Named Entity Recognition (FgNER) and reversible synthetic PII pseudonymization across 36 languages spoken by over 6.6 billion people. The system couples fine-grained multilingual sequence labeling with script-aware regex detectors to identify contextual entities (Person, Location, Organization, Medical) as well as structured technical PII (Emails, native-script Phone Numbers, IP Addresses, Credit Cards). AWED-PIPER offers a dual capability: full FgNER entity extraction and privacy-preserving reversible anonymization with persistent placeholders and de-anonymization dictionary mappings. The suite spans global languages to extremely low-resource vulnerable languages like Bodo, Manipuri, Bishnupriya, and Mizo. The resources can be accessed here: PII Protector Agentic Tool: (https://github.com/PrachuryyaKaushik/AWED-PIPER), FgNER Agentic Tool: (https://github.com/PrachuryyaKaushik/AWED-FiNER), PII Web Application: (https://hf.co/spaces/prachuryyaIITG/AWED_PII_Protector), FgNER Web Application: (https://hf.co/spaces/prachuryyaIITG/AWED-FiNER), and Edge-deployable Expert Detector Models: (https://hf.co/collections/prachuryyaIITG/awed-piper).
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。