arXiv:2603.05031cs.AI2026-03被引 1

检测AI生成界面的隐藏恶意行为,防止用户被误导或数据泄露。

AegisUI: Behavioral Anomaly Detection for Structured User Interface Protocols in AI Agent Systems

  • 通过生成带攻击的界面数据,提取18维特征进行异常检测
  • 随机森林模型在1000个恶意样本上达到93.1%准确率
  • 无需恶意样本训练的自编码器也表现良好,适合新系统部署

AI代理系统动态构建用户界面时,可能通过看似合法的结构化协议载荷实施欺骗:如按钮显示“查看账单”却会清空账户,或数据显示项暗中绑定内部薪资字段。现有防御仅检查语法,无法识别此类行为不一致问题。我们构建了AegisUI框架,生成4000个标注好的界面载荷(3000个良性,1000个恶意),涵盖五类应用领域和五种攻击类型(钓鱼界面、数据泄露、布局滥用、操纵性界面、工作流异常)。从每个载荷提取18个结构、语义、绑定和会话维度特征,对比三种检测器:孤立森林(无监督)、良性训练自编码器(半监督)和随机森林(有监督)。在80/20分层划分下,随机森林表现最佳(准确率0.931,精确率0.980,召回率0.740,F1 0.843,ROC-AUC 0.952);自编码器次之(F1 0.762,ROC-AUC 0.863),且无需恶意标签,适用于缺乏攻击历史的新系统。分析显示布局滥用最易检测,操纵性界面最难。所有代码、数据与配置均已开源。

原文摘要 · Abstract (English)

AI agents that build user interfaces on the fly assembling buttons, forms, and data displays from structured protocol payloads are becoming common in production systems. The trouble is that a payload can pass every schema check and still trick a user: a button might say "View invoice" while its hidden action wipes an account, or a display widget might quietly bind to an internal salary field. Current defenses stop at syntax; they were never built to catch this kind of behavioral mismatch. We built AegisUI to study exactly this gap. The framework generates structured UI payloads, injects realistic attacks into them, extracts numeric features, and benchmarks anomaly detectors end-to-end. We produced 4000 labeled payloads (3000 benign, 1000 malicious) spanning five application domains and five attack families: phishing interfaces, data leakage, layout abuse, manipulative UI, and workflow anomalies. From each payload we extracted 18 features covering structural, semantic, binding, and session dimensions, then compared three detectors: Isolation Forest (unsupervised), a benign-trained autoencoder (semi-supervised), and Random Forest (supervised). On a stratified 80/20 split, Random Forest scored best overall (accuracy 0.931, precision 0.980, recall 0.740, F1 0.843, ROC-AUC 0.952). The autoencoder came second (F1 0.762, ROC-AUC 0.863) and needs no malicious labels at training time, which matters when deploying a new system that lacks attack history. Per-attack-type analysis showed that layout abuse is easiest to catch while manipulative UI payloads are hardest. All code, data, and configurations are released for full reproducibility.

安全检测AI代理界面安全异常检测

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。