arXiv:2510.10990cs.CRcs.CL2025-10

提出新型隐私保护文本生成框架,精准防护敏感信息。

Secret-Protected Evolution for Differentially Private Synthetic Text Generation

  • 基于秘密感知的演化机制,动态区分敏感与非敏感内容
  • 在多个数据集上实现更低FID和更高任务准确率
  • 相比传统方法减少噪声需求,提升效率与实用性

文本数据在大型语言模型乃至通用人工智能发展中极为关键,但现实中大量高质量文本因隐私问题无法自由使用。为此,差分隐私(DP)合成文本生成被提出,旨在生成高价值合成数据的同时保护敏感信息。然而,现有方法采用统一保护策略,常对非敏感内容过度保护,导致显著效用损失与计算开销。本文提出秘密感知演化(SecPE)框架,扩展私有演化以支持秘密感知保护。理论上,SecPE满足$(π, γ)$-秘密保护,是高斯差分隐私的松弛形式,可在更优的效用-隐私权衡下运行,并大幅降低计算复杂度。实验表明,在OpenReview、PubMed和Yelp基准上,SecPE始终优于基于高斯DP的Aug-PE基线:取得更低弗雷谢特初始距离(FID)与更高下游任务准确率,且达到相同保护水平所需噪声更少。结果表明,秘密感知保护能显著提升隐私保护合成文本生成的实用性与有效性。

原文摘要 · Abstract (English)

Text data has become extremely valuable on large language models (LLMs) and even lead to general artificial intelligence (AGI). A lot of high-quality text in the real world is private and cannot be freely used due to privacy concerns. Therefore, differentially private (DP) synthetic text generation has been proposed, aiming to produce high-utility synthetic data while protecting sensitive information. However, existing DP synthetic text generation imposes uniform guarantees that often overprotect non-sensitive content, resulting in substantial utility loss and computational overhead. Therefore, we propose Secret-Protected Evolution (SecPE), a novel framework that extends private evolution with secret-aware protection. Theoretically, we show that SecPE satisfies $(\mathrm{p}, \mathrm{r})$-secret protection, constituting a relaxation of Gaussian DP that enables tighter utility-privacy trade-offs, while also substantially reducing computational complexity relative to baseline methods. Empirically, across the OpenReview, PubMed, and Yelp benchmarks, SecPE consistently achieves lower Fréchet Inception Distance (FID) and higher downstream task accuracy than GDP-based Aug-PE baselines, while requiring less noise to attain the same level of protection. Our results highlight that secret-aware guarantees can unlock more practical and effective privacy-preserving synthetic text generation.

隐私生成差分隐私文本合成高效算法

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。