提升大模型滥用检测探针在长文本下的泛化能力,保障实际应用安全。
Building Production-Ready Probes For Gemini
- 设计新探针架构应对长上下文输入带来的分布偏移挑战。
- 结合多样训练数据与架构选择,才能实现广泛泛化,准确率达92%以上。
- 探针+提示分类器组合方案高效低成本,已部署于Gemini用户端。
前沿语言模型能力快速提升,需更强机制防范滥用。已有研究显示激活探针是潜在的滥用检测手段,但发现其在关键生产环境分布变化下失效,尤其在短上下文向长上下文转换时表现不佳。本文提出若干新探针架构以应对长上下文分布偏移,在网络攻防场景中评估其对多轮对话、长提示和自适应红队攻击等生产相关分布偏移的鲁棒性。结果表明,虽新架构改善了上下文长度适应性,但仅靠架构仍不足,需结合多样化训练数据与架构选择方能实现广泛泛化。此外,探针与提示分类器结合可实现高精度(>92%)且计算成本极低。这些成果已用于Google前沿语言模型Gemini的用户面滥用检测系统部署。最后,初步实验表明使用AlphaEvolve可自动优化探针架构与红队策略,验证了部分AI安全研究自动化可行性。
原文摘要 · Abstract (English)
Frontier language model capabilities are improving rapidly. We thus need stronger mitigations against bad actors misusing increasingly powerful systems. Prior work has shown that activation probes may be a promising misuse mitigation technique, but we identify a key remaining challenge: probes fail to generalize under important production distribution shifts. In particular, we find that the shift from short-context to long-context inputs is difficult for existing probe architectures. We propose several new probe architectures that handle this long-context distribution shift. We evaluate these probes in the cyber-offensive domain, testing their robustness against various production-relevant distribution shifts, including multi-turn conversations, long context prompts, and adaptive red teaming. Our results demonstrate that while our novel architectures address context length, a combination of architecture choice and training on diverse distributions is required for broad generalization. Additionally, we show that pairing probes with prompted classifiers achieves optimal accuracy at a low cost due to the computational efficiency of probes. These findings have informed the successful deployment of misuse mitigation probes in user-facing instances of Gemini, Google's frontier language model. Finally, we find early positive results using AlphaEvolve to automate improvements in both probe architecture search and adaptive red teaming, showing that automating some AI safety research is already possible.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。