在强隐私保护下生成高质量可控文本，提升合成数据实用性和指令遵循能力。
ACTG-ARL: Differentially Private Conditional Text Generation with RL-Boosted Control
- 分层框架将生成任务拆解为特征学习与条件生成，提升可控性。
- 相比之前方法，合成文本质量提升20%（MAUVE指标），隐私保障更强。
- 引入锚定强化学习，避免奖励滥用，适合需要精准控制的生成场景。
在差分隐私（DP）约束下生成高质量合成文本对语言模型训练与评估至关重要。现有方法常无法保持关键统计特性，因添加噪声导致性能下降，且缺乏细粒度控制。为此，本文提出双阶段框架：先通过表结构特征学习，再用隐私保护的条件生成器生成文本，形成ACTG（属性条件文本生成）。进一步提出锚定强化学习（ARL），结合强化学习增强控制力，并以SFT最佳-N数据作为锚点防止奖励劫持。整体算法ACTG-ARL在强隐私保护下实现合成文本质量提升20%（MAUVE指标），显著改善指令遵循能力。
原文摘要 · Abstract (English)
Generating high-quality synthetic text under differential privacy (DP) is critical for training and evaluating language models without compromising user privacy. Prior work on synthesizing DP datasets often fail to preserve key statistical attributes, suffer utility loss from the noise required by DP, and lack fine-grained control over generation. To address these challenges, we make two contributions. First, we introduce a hierarchical framework that decomposes DP synthetic text generation into two subtasks: feature learning and conditional text generation. This design explicitly incorporates learned features into the generation process and simplifies the end-to-end synthesis task. Through systematic ablations, we identify the most effective configuration: a rich tabular schema as feature, a DP tabular synthesizer, and a DP fine-tuned conditional generator, which we term ACTG (Attribute-Conditioned Text Generation). Second, we propose Anchored RL (ARL), a post-training method that improves the instruction-following ability of ACTG for conditional generation. ARL combines RL to boost control with an SFT anchor on best-of-$N$ data to prevent reward hacking. Together, these components form our end-to-end algorithm ACTG-ARL, which advances both the quality of DP synthetic text (+20% MAUVE over prior work) and the control of the conditional generator under strong privacy guarantees.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。