arXiv:2512.00303cs.LGcs.AI2025-12

提出RGIA攻击法,破解联邦强化学习中的隐私泄露问题。

Gradient Inversion in Federated Reinforcement Learning

  • 用先验知识正则化状态、奖励和动态,约束重建数据分布。
  • 在控制与自动驾驶任务中成功重构私有训练数据。
  • 适合关注联邦学习隐私安全的研究者与从业者。

联邦强化学习(FRL)通过共享梯度实现分布式策略优化,同时保护本地数据隐私。然而,该方法存在隐私泄露风险:攻击者可利用共享梯度重构局部训练数据。与传统监督联邦学习不同,FRL中的数据重构不仅需匹配梯度,还需符合环境的真实转移动态(即真实数据转移分布)。为此,本文提出新型攻击方法——正则化梯度反演攻击(RGIA),在优化过程中对状态、奖励和转移动态施加基于先验知识的正则化,确保重建数据贴近真实转移分布。理论证明,该正则化项将解空间从包含伪解的广泛集合缩小至满足梯度匹配与真实动态的受限子集。大量实验在控制任务与自动驾驶任务上表明,RGIA能有效约束重建数据的转移分布,从而成功重构局部私有数据。

原文摘要 · Abstract (English)

Federated reinforcement learning (FRL) enables distributed learning of optimal policies while preserving local data privacy through gradient sharing.However, FRL faces the risk of data privacy leaks, where attackers exploit shared gradients to reconstruct local training data.Compared to traditional supervised federated learning, successful reconstruction in FRL requires the generated data not only to match the shared gradients but also to align with real transition dynamics of the environment (i.e., aligning with the real data transition distribution).To address this issue, we propose a novel attack method called Regularization Gradient Inversion Attack (RGIA), which enforces prior-knowledge-based regularization on states, rewards, and transition dynamics during the optimization process to ensure that the reconstructed data remain close to the true transition distribution.Theoretically, we prove that the prior-knowledge-based regularization term narrows the solution space from a broad set containing spurious solutions to a constrained subset that satisfies both gradient matching and true transition dynamics.Extensive experiments on control tasks and autonomous driving tasks demonstrate that RGIA can effectively constrain reconstructed data transition distributions and thus successfully reconstruct local private data.

联邦学习隐私安全强化学习梯度反演

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。