arXiv:2510.16054cs.CRcs.CL2025-10ACL被引 6

用强化学习动态决定哪些数据该本地处理,哪些该远程发送,兼顾隐私与性能。

Privacy-R1: Privacy-Aware Multi-LLM Agent Collaboration via Reinforcement Learning

  • 通过强化学习训练智能体,动态分配文本片段处理任务。
  • 在医疗数据集上实现隐私与任务性能的新平衡点。
  • 适合需要保护敏感信息的高安全场景使用。

当用户向大语言模型提交查询时,提示词常包含敏感信息,导致两难:将查询发送给强大但可能泄露数据的专有模型,或依赖本地小模型以保障隐私但降低性能。现有方法依赖静态重写管道,破坏语言连贯性并盲目移除所有敏感信息,包括任务关键内容。本文将此问题重构为序列决策问题,提出名为Privacy-R1的新型强化学习框架。该框架训练智能体动态路由文本块,学习最优策略,在隐私泄露与任务性能间取得平衡。它能隐式区分可替换的个人身份信息(本地屏蔽)与任务关键的个人身份信息(战略性发送至远程模型以最大化效用)。为验证复杂场景下的效果,我们引入一个高敏感信息密度的新医学数据集。实验表明,该框架在隐私-效用前沿达到新最佳水平,证明了学习型自适应策略在敏感环境部署LLM中的必要性。数据集地址:https://github.com/zackhuiiiii/Privacy-R1。

原文摘要 · Abstract (English)

When users submit queries to Large Language Models (LLMs), their prompts can often contain sensitive data, forcing a difficult choice: Send the query to a powerful proprietary LLM providers to achieving state-of-the-art performance and risk data exposure, or relying on smaller, local models guarantees data privacy but often results in a degradation of task performance. Prior approaches have relied on static pipelines that use LLM rewriting, which shatters linguistic coherence and indiscriminately removes privacy-sensitive information, including task-critical content. We reformulate this challenge (Privacy-Conscious Delegation) as a sequential decision-making problem and introduce a novel reinforcement learning (RL) framework called Privacy-R1 to solve it. Our framework trains an agent to dynamically route text chunks, learning a policy that optimally balances the trade-off between privacy leakage and task performance. It implicitly distinguishes between replaceable Personally Identifiable Information (PII) (which it shields locally) and task-critical PII (which it strategically sends to the remote model for maximal utility). To validate our approach in complex scenarios, we also introduce a new medical dataset with high PII density. Our framework achieves a new state-of-the-art on the privacy-utility frontier, demonstrating the necessity of learned, adaptive policies for deploying LLMs in sensitive environments. Dataset can be found at: https://github.com/zackhuiiiii/Privacy-R1.

隐私保护强化学习LLM协作数据安全

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。