用图神经网络分析大模型工具调用流量，识别攻击行为。
Content-Aware Attack Detection in LLM Agent Tool-Call Traffic: An Empirical Study of Features, Architectures, and Evaluation Protocols
- 将工具调用建模为带内容嵌入的图结构，捕捉调用间依赖关系。
- 结合语义嵌入特征后检测准确率（AUROC）超0.89，远高于仅用元数据。
- 发现任务无关划分会夸大效果26个百分点，需谨慎评估。
大语言模型代理通过模型上下文协议（MCP）调用外部工具已成主流，但对其工具调用流量的智能监控仍缺乏深入研究。本文提出一种基于图神经网络的攻击检测框架：将每个代理会话表示为图结构（工具调用为节点，时序与数据流为边），并用SBERT对参数和响应进行语义嵌入。对比了三种GNN架构（GAT、GCN、GraphSAGE）、无图的MLP及经典分类器（XGBoost、随机森林、逻辑回归、线性SVM）。在RAS-Eval（任务分层划分）上全面评估，另在ATBench和联合源变体（标签分层划分）中保留GraphSAGE作为基准。结果表明：第一，内容特征至关重要——仅使用元数据时AUROC稳定在0.64，而引入语义嵌入后提升至0.89以上；第二，朴素随机划分可使AUROC虚高26个百分点，存在记忆泄露问题，此前工作未予关注；第三，检测信号主要来自语义嵌入——树集成模型在池化嵌入上达到0.975的AUROC，优于主设置中的各神经架构（如GNN为0.917，MLP为0.896），且自监督预训练在此任务上未体现标签效率优势。
原文摘要 · Abstract (English)
The Model Context Protocol (MCP) has become a widely adopted interface for LLM agents to invoke external tools, yet learned monitoring of MCP tool-call traffic remains underexplored. In this article, the proposed detector is presented as an attack detection framework for MCP tool-call traffic that encodes each agent session as a graph (tool calls as nodes, sequential and data-flow links as edges), enriches nodes with sentence-embedding features over arguments and responses, and classifies sessions as benign or attacked. Three GNN architectures (GAT, GCN, GraphSAGE), a no-graph MLP, and classical baselines (XGBoost, random forest, logistic regression, linear SVM) are evaluated, with the full architecture comparison conducted on RAS-Eval (task-stratified splits) and GraphSAGE retained as the GNN baseline on ATBench and a combined-source variant (both label-stratified). Three findings emerge. First, content-level features are essential: metadata-only detection plateaus around an AUROC of 0.64 regardless of architecture, while content embeddings push the AUROC above 0.89. Second, naive random-split evaluation inflates AUROC by up to 26 percentage points relative to task-disjoint splits, a memorization confound that prior agent-detection work has not addressed. Third, the detection signal resides primarily in the SBERT content embeddings: an AUROC of 0.975 was reached by tree ensembles on pooled embeddings, performing, for the most part, better than the neural architectures in the primary RAS-Eval setting including GNNs (0.917) and the MLP (0.896), and self-supervised pre-training does not deliver a label-efficiency advantage on this task.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。