arXiv:2503.00596cs.CLcs.AI2025-03ICLR被引 20

攻击者可让大模型评分系统偏袒自己,只需少量数据污染即可大幅提高得分。

BadJudge: Backdoor Vulnerabilities of LLM-as-a-Judge

  • 通过污染1%的评分模型训练数据,用单个词触发后门,使攻击者得分翻倍。
  • 在最弱攻击条件下(网页污染)仍能提升20%评分,强攻击下得分从1.5升至4.9。
  • 该攻击对多种模型、任务和触发方式均有效,适合关注模型评估安全的研究者。

本文提出一种新型后门攻击,针对大模型作为评分器的评估体系,攻击者同时控制候选模型和评分模型。通过仅污染1%的评分模型训练数据,用一个简单令牌触发后门,即可使攻击者得分相对于合法得分翻三倍。研究系统分类了三种现实场景下的数据访问级别:(1)网页污染,(2)恶意标注者,(3)权重污染,其攻击强度逐级增强。在最弱假设下(网页污染),攻击仍可造成20%评分膨胀;在最强假设下(权重污染),攻击者得分可从1.5/5提升至4.9/5。该攻击具备跨架构、跨任务、跨触发设计的泛化能力。当污染10%训练数据时,可使毒性检测器误判有毒提示为无害达89%,使RAG中的文档重排序器将中毒文档排第一达97%。由于大模型评分系统处于伦理与技术交汇点,防御手段受限。模型融合被证明是有效缓解手段,可在保持顶级性能的同时将攻击成功率降至接近0%,且计算成本低、易集成到现有训练流程中。

原文摘要 · Abstract (English)

This paper proposes a novel backdoor threat attacking the LLM-as-a-Judge evaluation regime, where the adversary controls both the candidate and evaluator model. The backdoored evaluator victimizes benign users by unfairly assigning inflated scores to adversary. A trivial single token backdoor poisoning 1% of the evaluator training data triples the adversary's score with respect to their legitimate score. We systematically categorize levels of data access corresponding to three real-world settings, (1) web poisoning, (2) malicious annotator, and (3) weight poisoning. These regimes reflect a weak to strong escalation of data access that highly correlates with attack severity. Under the weakest assumptions - web poisoning (1), the adversary still induces a 20% score inflation. Likewise, in the (3) weight poisoning regime, the stronger assumptions enable the adversary to inflate their scores from 1.5/5 to 4.9/5. The backdoor threat generalizes across different evaluator architectures, trigger designs, evaluation tasks, and poisoning rates. By poisoning 10% of the evaluator training data, we control toxicity judges (Guardrails) to misclassify toxic prompts as non-toxic 89% of the time, and document reranker judges in RAG to rank the poisoned document first 97% of the time. LLM-as-a-Judge is uniquely positioned at the intersection of ethics and technology, where social implications of mislead model selection and evaluation constrain the available defensive tools. Amidst these challenges, model merging emerges as a principled tool to offset the backdoor, reducing ASR to near 0% whilst maintaining SOTA performance. Model merging's low computational cost and convenient integration into the current LLM Judge training pipeline position it as a promising avenue for backdoor mitigation in the LLM-as-a-Judge setting.

模型安全后门攻击评分系统LLM评估

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。