用提示词驱动的LLM对激活探针做轻量级红队测试，发现其真实场景下的脆弱性。
Red-teaming Activation Probes using Prompted LLMs
- 用现成LLM+迭代反馈和上下文学习构建无须微调的黑盒红队方法
- 发现探针在法律术语下误报、程序化语气下漏报等可解释的脆弱模式
- 无需修改模型即可提前暴露风险，适合安全评估与探针优化
激活探针因低成本和低延迟而成为AI系统监控的有力工具，但其在真实环境中的鲁棒性尚未充分探索。本文提出一种轻量级黑盒红队方法，通过将现成LLM与迭代反馈和上下文学习（ICL）结合，无需微调、梯度或架构访问即可进行对抗性测试。以高风险交互场景为例，该方法揭示了当前SOTA探针的关键缺陷：法律术语导致误报（FPs），程式化语调引发漏报（FNs），且在情景约束攻击下仍存在持续漏洞。结果表明，仅通过提示词框架即可提前识别失效模式，为未来探针的加固提供可操作的洞察。
原文摘要 · Abstract (English)
Activation probes are attractive monitors for AI systems due to low cost and latency, but their real-world robustness remains underexplored. We ask: What failure modes arise under realistic, black-box adversarial pressure, and how can we surface them with minimal effort? We present a lightweight black-box red-teaming procedure that wraps an off-the-shelf LLM with iterative feedback and in-context learning (ICL), and requires no fine-tuning, gradients, or architectural access. Running a case study with probes for high-stakes interactions, we show that our approach can help discover valuable insights about a SOTA probe. Our analysis uncovers interpretable brittleness patterns (e.g., legalese-induced FPs; bland procedural tone FNs) and reduced but persistent vulnerabilities under scenario-constraint attacks. These results suggest that simple prompted red-teaming scaffolding can anticipate failure patterns before deployment and might yield promising, actionable insights to harden future probes.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。