首次验证大模型间行为后门检测的泛化能力，发现通用检测效果差但加模型身份可显著提升。
Cross-LLM Generalization of Behavioral Backdoor Detection in AI Agent Supply Chains
- 通过跨模型对比，发现单模型检测器在不同大模型间准确率下降超40个百分点。
- 检测失败主因是时序特征差异大（变异系数>0.8），结构特征则相对稳定。
- 加入模型身份信息后，检测准确率普遍达90.6%，适合多模型部署企业使用。
随着AI代理融入企业工作流，其对共享工具库和预训练组件的依赖带来供应链安全风险。尽管已有研究在单一大模型中实现行为后门检测，但跨模型泛化问题尚未被探索，这对部署多系统的企业具有严重隐患。本文首次系统性研究跨大模型行为后门检测，评估了六种生产级大模型（GPT-5.1、Claude Sonnet 4.5、Grok 4.1、Llama 4 Maverick、GPT-OSS 120B、DeepSeek Chat V3.1）间的泛化能力。基于1,198条执行轨迹与36次跨模型实验，我们发现：单模型检测器在其训练分布内准确率达92.7%，但在不同模型间降至49.2%，泛化差距达43.4个百分点，几乎等同于随机猜测。分析表明，该差距源于模型特有的行为特征，尤其体现在时序特征上（变异系数 > 0.8），而结构特征在不同架构间保持稳定。我们提出一种引入模型身份作为附加特征的检测方法，在所有测试模型上均实现90.6%的准确率。研究数据集与框架已公开，支持可复现研究。
原文摘要 · Abstract (English)
As AI agents become integral to enterprise workflows, their reliance on shared tool libraries and pre-trained components creates significant supply chain vulnerabilities. While previous work has demonstrated behavioral backdoor detection within individual LLM architectures, the critical question of cross-LLM generalization remains unexplored, a gap with serious implications for organizations deploying multiple AI systems. We present the first systematic study of cross-LLM behavioral backdoor detection, evaluating generalization across six production LLMs (GPT-5.1, Claude Sonnet 4.5, Grok 4.1, Llama 4 Maverick, GPT-OSS 120B, and DeepSeek Chat V3.1). Through 1,198 execution traces and 36 cross-model experiments, we quantify a critical finding: single-model detectors achieve 92.7% accuracy within their training distribution but only 49.2% across different LLMs, a 43.4 percentage point generalization gap equivalent to random guessing. Our analysis reveals that this gap stems from model-specific behavioral signatures, particularly in temporal features (coefficient of variation > 0.8), while structural features remain stable across architectures. We show that model-aware detection incorporating model identity as an additional feature achieves 90.6% accuracy universally across all evaluated models. We release our multi-LLM trace dataset and detection framework to enable reproducible research.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。