arXiv:2602.00350cs.CV2026-02

用强化学习自动找回被删除的图像概念,效率高且可复用。

ReLAPSe: Reinforcement-Learning-trained Adversarial Prompt Search for Erased concepts in unlearned diffusion models

  • 将概念恢复建模为强化学习问题,用噪声预测损失做反馈信号
  • 在多个主流去学习方法上实现近实时的精细身份与风格恢复
  • 适合安全评估人员测试扩散模型去学习效果,尤其敏感内容

机器去学习是移除文本到图像扩散模型中未经授权概念的关键防御机制,但最新研究表明,去学习后潜在视觉信息仍可能残留。现有对抗性方法存在根本局限:基于优化的方法因逐实例迭代搜索而计算成本高;基于推理和启发式的技术则缺乏目标模型潜在视觉表示的直接反馈。为此,我们提出 ReLAPSe,一种基于策略的对抗框架,将概念恢复重构为强化学习问题。ReLAPSe 使用带可验证奖励的强化学习(RLVR),以扩散模型的噪声预测损失作为模型内生且可验证的反馈信号。该闭环设计直接将文本提示操作与潜在视觉残余对齐,使智能体学习可迁移的恢复策略,而非优化孤立提示。通过从逐实例优化转向全局策略学习,ReLAPSe 实现了高效、近实时的细粒度身份与风格恢复,适用于多种先进去学习方法,为严格红队测试去学习扩散模型提供可扩展工具。部分实验涉及敏感视觉概念,如裸露。代码已公开于 https://github.com/gmum/ReLaPSe。

原文摘要 · Abstract (English)

Machine unlearning is a key defense mechanism for removing unauthorized concepts from text-to-image diffusion models, yet recent evidence shows that latent visual information often persists after unlearning. Existing adversarial approaches for exploiting this leakage are constrained by fundamental limitations: optimization-based methods are computationally expensive due to per-instance iterative search. At the same time, reasoning-based and heuristic techniques lack direct feedback from the target model's latent visual representations. To address these challenges, we introduce ReLAPSe, a policy-based adversarial framework that reformulates concept restoration as a reinforcement learning problem. ReLAPSe trains an agent using Reinforcement Learning with Verifiable Rewards (RLVR), leveraging the diffusion model's noise prediction loss as a model-intrinsic and verifiable feedback signal. This closed-loop design directly aligns textual prompt manipulation with latent visual residuals, enabling the agent to learn transferable restoration strategies rather than optimizing isolated prompts. By pioneering the shift from per-instance optimization to global policy learning, ReLAPSe achieves efficient, near-real-time recovery of fine-grained identities and styles across multiple state-of-the-art unlearning methods, providing a scalable tool for rigorous red-teaming of unlearned diffusion models. Some experimental evaluations involve sensitive visual concepts, such as nudity. Code is available at https://github.com/gmum/ReLaPSe

扩散模型去学习对抗攻击强化学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。