发现简历筛选中大模型易受隐蔽指令攻击，提出有效防御方案。
AI Security Beyond Core Domains: Resume Screening as a Case Study of Adversarial Vulnerabilities in Specialized LLM Applications
- 通过构建基准测试，揭示简历筛选中80%以上攻击成功率。
- 提出FIDS防御机制，比传统方法减少15.4%攻击，误拒率仅增10.4%。
- 训练阶段防御优于推理阶段，兼顾安全性与实用性。
大语言模型在文本理解与生成方面表现优异，常用于代码审查、内容审核等自动化任务。然而，我们的研究发现，模型易受隐藏在输入数据（如简历或代码）中的“对抗性指令”影响，导致行为偏离预期。值得注意的是，尽管成熟领域如代码审查已有防御措施，但在简历筛选、同行评审等常见应用中往往缺乏防护。本文针对简历筛选场景构建了评估基准，结果显示某些攻击类型的成功率超过80%。我们评估了两种防御机制：基于提示的防御使攻击减少10.1%，但误拒率上升12.5%；而提出的FIDS（通过LoRA适配实现外来指令检测）方法可实现15.4%的攻击减少，误拒率仅增加10.4%。二者结合可实现26.3%的攻击减少，表明训练时防御在安全性和可用性上均优于推理时缓解。
原文摘要 · Abstract (English)
Large Language Models (LLMs) excel at text comprehension and generation, making them ideal for automated tasks like code review and content moderation. However, our research identifies a vulnerability: LLMs can be manipulated by "adversarial instructions" hidden in input data, such as resumes or code, causing them to deviate from their intended task. Notably, while defenses may exist for mature domains such as code review, they are often absent in other common applications such as resume screening and peer review. This paper introduces a benchmark to assess this vulnerability in resume screening, revealing attack success rates exceeding 80% for certain attack types. We evaluate two defense mechanisms: prompt-based defenses achieve 10.1% attack reduction with 12.5% false rejection increase, while our proposed FIDS (Foreign Instruction Detection through Separation) using LoRA adaptation achieves 15.4% attack reduction with 10.4% false rejection increase. The combined approach provides 26.3% attack reduction, demonstrating that training-time defenses outperform inference-time mitigations in both security and utility preservation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。