针对大模型提取攻击设计新方法,提升窃取效率与隐蔽性。
"Yes, My LoRD." Guiding Language Model Extraction with Locality Reinforced Distillation
- 用策略梯度思想引导本地模型学习目标模型偏好
- 实验显示可降低查询次数并绕过水印保护
- 适合研究模型安全与对抗攻击的学者使用
大语言模型(LLM)的模型提取攻击(MEA)近年受到广泛关注。然而,现有方法多沿用深度神经网络(DNN)的攻击策略,忽略了提取任务与LLM对齐训练之间的本质差异,导致攻击效果不佳。为此,本文提出专为LLM设计的局部强化蒸馏(LoRD)算法。LoRD引入一种新的基于策略梯度的训练任务,利用目标模型的响应作为信号,指导本地模型生成符合其偏好的输出。理论分析表明:I)LoRD的收敛过程与LLM对齐过程一致;II)通过探索式窃取,可在降低查询复杂度的同时有效规避水印保护。大量实验验证了该方法在提取多个先进商用大模型时的优越性。代码已开源:https://github.com/liangzid/LoRD-MEA。
原文摘要 · Abstract (English)
Model extraction attacks (MEAs) on large language models (LLMs) have received increasing attention in recent research. However, existing attack methods typically adapt the extraction strategies originally developed for deep neural networks (DNNs). They neglect the underlying inconsistency between the training tasks of MEA and LLM alignment, leading to suboptimal attack performance. To tackle this issue, we propose Locality Reinforced Distillation (LoRD), a novel model extraction algorithm specifically designed for LLMs. In particular, LoRD employs a newly defined policy-gradient-style training task that utilizes the responses of victim model as the signal to guide the crafting of preference for the local model. Theoretical analyses demonstrate that I) The convergence procedure of LoRD in model extraction is consistent with the alignment procedure of LLMs, and II) LoRD can reduce query complexity while mitigating watermark protection through our exploration-based stealing. Extensive experiments validate the superiority of our method in extracting various state-of-the-art commercial LLMs. Our code is available at: https://github.com/liangzid/LoRD-MEA .
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。