小模型生成假信息替代敏感数据，避免重复输出且更自然。
Locale-Conditioned Few-Shot Prompting Mitigates Demonstration Regurgitation in On-Device PII Substitution with Small Language Models
- 用地理位置动态选样例，防止小模型照搬示例内容。
- 生成的假名等在6个语种中表现优于规则生成器，文本长度更接近原文。
- 适合注重生成自然性的隐私保护场景，但训练数据多样性会下降。
个人身份信息（PII）脱敏通常用[PERSON]等占位符替换，破坏下游检索与命名实体识别（NER）的可用性。我们提出全本地化流程：1.5B混合专家分类器（openai/privacy-filter）检测实体，1比特Bonsai-1.7B小语言模型（SLM）生成上下文相关的假名、地址和日期，规则生成器（faker）处理有规律字段。发现提示工程比量化更重要：使用固定三样本提示时，1比特SLM会原样复现示例；1.58比特三值版也出现完全相同的错误，排除了量化影响。采用基于字符范围的地域纯池与输入MD5哈希采样三例的新方法后，482次调用全部成功，无复读现象，生成符合地域的假数据。尽管仍依赖少量同地域示例，其残余局限可量化。在2000文档多语言语料上，混合困惑度（PPL）在六个语种均优于faker，长度保留最佳者达4/6。下游NER任务（400训练 / 100测试，英文）中，直接脱敏F1=0.000，faker为0.656，原始数据为0.960；在匹配子集（160/40）中，faker（0.506）显著优于混合方案（0.346，p<0.001）。此为真实负面结果：尽管生成文本更自然，但训练分布变差，多样性对下游效果更重要。
原文摘要 · Abstract (English)
Personally Identifiable Information (PII) redaction usually replaces detected entities with placeholder tokens such as [PERSON], destroying the downstream utility of the redacted text for retrieval and Named Entity Recognition (NER) training. We propose a fully on-device pipeline that substitutes PII with consistent, type-preserving fake values: a 1.5 B mixture-of-experts token classifier (openai/privacy-filter) detects spans, a 1-bit Bonsai-1.7B Small Language Model (SLM) proposes contextual surrogates for names, addresses, and dates, and a rule-based generator (faker) handles patterned fields. We report a prompting finding more important than the quantization choice: with naive fixed three-shot demonstrations, the 1-bit SLM regurgitates demonstration outputs verbatim regardless of input; 1.58-bit Ternary-Bonsai-1.7B reproduces byte-identical failures, ruling out quantization as the cause. We fix this with locale-conditioned rotating few-shot demonstrations: a character-range heuristic picks a locale-pure pool and a per-input MD5 hash samples three demonstrations. With the fix, 482/482 unique Bonsai-1.7B calls succeed (no echoes) and produce locale-correct surrogates, although the SLM still copies from a small same-locale demonstration pool - a residual narrowness we quantify. On a 2000-document multilingual corpus, hybrid perplexity (PPL) beats faker in all six locales under a multilingual evaluator (XGLM-564M); length preservation is best-of-three in 4 of 6 locales. On downstream NER (400 train / 100 test, English), redact yields F1=0.000, faker 0.656, original 0.960; on a matched 160/40 subset including hybrid, faker (0.506) outperforms hybrid (0.346) at p < 0.001. We report this as an honest negative finding: SLM surrogates produce more natural text but a less varied training distribution, and downstream NER benefits more from variety than from naturalness.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。