用强化学习精准提取大模型中的敏感数据，突破现有方法局限。
DMRL: Data- and Model-aware Reward Learning for Data Extraction
- 通过逆强化学习构建推理数据集，捕捉泄露思维模式
- 在多个模型上实现比基线方法更高的数据提取成功率
- 动态调整优化策略，适应不同数据与模型难度
大语言模型易受隐私泄露威胁，系统性红队测试对发展稳健防御机制至关重要。然而，现有数据提取方法存在三大缺陷：（1）依赖数据集重复（可通过去重解决），（2）依赖提示工程（现已被检测与防御手段应对），（3）依赖随机搜索生成对抗样本。为此，我们提出 DMRL——一种数据与模型感知的奖励学习方法，用于数据提取。该方法采用逆强化学习，包含两个核心组件：（1）构建内省式推理数据集，以捕捉数据泄露思维模式并引导模型行为；（2）使用分组相对策略优化（GRPO）训练奖励模型，根据任务在数据与模型层面的难度动态调整优化策略。在多种大语言模型上的全面实验表明，DMRL 在数据提取性能上优于所有基线方法。
原文摘要 · Abstract (English)
Large language models (LLMs) are inherently vulnerable to unintended privacy breaches. Consequently, systematic red-teaming research is essential for developing robust defense mechanisms. However, current data extraction methods suffer from several limitations: (1) rely on dataset duplicates (addressable via deduplication), (2) depend on prompt engineering (now countered by detection and defense), and (3) rely on random-search adversarial generation. To address these challenges, we propose DMRL, a Data- and Model-aware Reward Learning approach for data extraction. This technique leverages inverse reinforcement learning to extract sensitive data from LLMs. Our method consists of two main components: (1) constructing an introspective reasoning dataset that captures leakage mindsets to guide model behavior, and (2) training reward models with Group Relative Policy Optimization (GRPO), dynamically tuning optimization based on task difficulty at both the data and model levels. Comprehensive experiments across various LLMs demonstrate that DMRL outperforms all baseline methods in data extraction performance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。