通过频率分析定位代码大模型的恶意输出与后门触发器
Localizing Malicious Outputs from CodeLLM
- 基于变异输入的频率排名，识别恶意子串及其触发条件
- 在98%情况下将恶意输出排进前五，平均攻击成功率86.6%
- 对少量触发样本仍有效，比现有方法提升35%-50%
我们提出FreqRank，一种基于变异的防御方法，用于定位大语言模型输出中的恶意组件及其对应的后门触发器。该方法假设恶意子串在触发输入下会持续出现，并通过频率排名系统识别这些内容，进而定位输入中的后门触发器。我们针对代码补全（CC）、代码生成（CG）和代码摘要（CS）三个下游任务，通过微调或自定义指令构建了九个恶意模型，其平均攻击成功率（ASR）达86.6%。实验表明，FreqRank在98%的情况下将恶意输出列为前五候选。此外，随着变异样本数量增加，其效果持续提升，即使在触发样本有限时仍能有效定位后门触发器。最终，该方法比其他防御手段高出35%-50%的有效性。
原文摘要 · Abstract (English)
We introduce FreqRank, a mutation-based defense to localize malicious components in LLM outputs and their corresponding backdoor triggers. FreqRank assumes that the malicious sub-string(s) consistently appear in outputs for triggered inputs and uses a frequency-based ranking system to identify them. Our ranking system then leverages this knowledge to localize the backdoor triggers present in the inputs. We create nine malicious models through fine-tuning or custom instructions for three downstream tasks, namely, code completion (CC), code generation (CG), and code summarization (CS), and show that they have an average attack success rate (ASR) of 86.6%. Furthermore, FreqRank's ranking system highlights the malicious outputs as one of the top five suggestions in 98% of cases. We also demonstrate that FreqRank's effectiveness scales as the number of mutants increases and show that FreqRank is capable of localizing the backdoor trigger effectively even with a limited number of triggered samples. Finally, we show that our approach is 35-50% more effective than other defense methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。