用简单方法检测大模型接口中的模型窃取攻击
An Embarrassingly Simple Detector for Model Extraction Attacks in Large Language Model API Traffic
- 通过语义空间分布对比，判断流量是否异常
- 在多种攻击场景下误报率仅0.3%，检出率达100%
- 适合保护API服务的开发者和安全团队使用
大型语言模型（LLMs）越来越多地通过托管API部署，使模型窃取成为实际威胁。单个窃取查询常与正常请求难以区分，现有方法多聚焦于单次查询异常评分或纯良性/攻击者用户设定。本文将模型窃取监测建模为良性质心流量窗口分布检测：将输入查询嵌入语义空间，测试其聚合分布是否偏离历史良性流量。采用最大均值差异（MMD）实现该方法，仅通过良性对良性比较确定决策阈值。在四个窃取场景下的十四组攻击-正常查询对上评估，对比了改进的PRADA、SEAT、CAP、DATE、边际马氏距离及伪类别能量基线。在三个随机种子下，MMD实现0.3%良性误报率、100.0%纯攻击者召回率、攻击占比下的90.5%平均召回率以及95.1%平衡准确率。结果表明，良性质心分布检测是用户级与混合多用户场景下模型窃取检测的强效基准。
原文摘要 · Abstract (English)
Large language models (LLMs) are increasingly deployed through hosted APIs, making model extraction a practical threat to model ownership and service security. Individual extraction queries often resemble benign requests, while existing evaluations often focus on single-query anomaly scoring or pure benign-versus-attacker user settings. We formulate model extraction monitoring as benign-calibrated traffic-window distribution testing: embed incoming queries into a semantic space and test whether their aggregate distribution deviates from historical benign traffic. We instantiate this formulation with maximum mean discrepancy (MMD), using only benign-vs-benign comparisons to set the decision threshold. We evaluate on fourteen attacker-normal query pairs from four extraction scenarios and compare with adapted PRADA, SEAT, CAP, DATE, marginal Mahalanobis, and pseudo-class energy baselines. Across three random seeds, MMD achieves 0.3% benign FPR, 100.0% pure-attacker TPR, 90.5% average TPR over attacker fractions, and 95.1% balanced accuracy. These results show that benign-calibrated distribution testing is a strong empirical baseline for model extraction detection in both user-level and mixed multi-user LLM API traffic.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。