用CPU分类器+分阶段流水线,大幅降低大模型安全检测的算力成本。
Do You Really Need a GPU to Guard Your LLM? CPU-Class Classifiers and Multi-Stage Pipelines for Safety Enforcement at Scale
- 设计多阶段流水线,用CPU分类器处理大部分常规请求
- 在对抗性输入下,CPU分类器比GPU模型高26个百分点的准确率
- 80%的正常请求可在低成本CPU阶段解决,适合大规模部署
用于检测大模型输入中越狱尝试的安全分类器已成为标准部署组件,但几乎所有系统依赖于基于GPU的模型(如微调过的Transformer和大模型作为裁判)。这些方法带来显著的每查询延迟与基础设施开销。很少有研究探讨基于CPU的分类器(如支持向量机、梯度提升树)在TF-IDF特征上能否在真实生产环境中达到同等精度。本文评估了五类CPU分类器、基于SSM的Mamba-130M以及两种基于Transformer的GPU模型(DeBERTa-v3与Gemma-2B LoRA),覆盖九种越狱来源和三种场景:分布内(D1)、分布外(D2)及对抗混淆(D3)。在D1场景下,最优CPU分类器以约五分之一的部署成本达到最佳变压器模型的性能;在D2场景中,CPU分类器因自信误校准而失败,产生高置信度的漏检;在D3场景中,其F1值超过变压器模型26个百分点以上。基于互补的失效模式,本文提出GuardChain三阶段安全流水线(正则表达式→CPU→GPU),将每个提示路由至最廉价且能做出置信判断的阶段。仅CPU阶段即可在接近峰值精度下处理80%的分布内请求,而GPU阶段则补救分布外的失败。对大规模部署大模型安全性的从业者而言,本工作表明多数流量无需依赖GPU级基础设施。
原文摘要 · Abstract (English)
Safety classifiers that screen LLM inputs for jailbreak attempts have become standard deployment components, yet almost all production systems rely on GPU-based models: fine-tuned transformers and LLM-as-a-judge pipelines. These approaches impose significant per-query latency and infrastructure cost. Very little research has asked whether CPU-based classifiers, such as support vector machines and gradient-boosted trees trained on TF-IDF features, can match their accuracy across the conditions that production deployments encounter. We evaluate five CPU classifier families, Mamba-130M as an SSM-based GPU classifier, and transformer-based GPU models (DeBERTa-v3 and Gemma-2B with LoRA) across nine jailbreak sources and three regimes: in-distribution (D1), out-of-distribution (D2), and adversarially obfuscated (D3). On D1, the best CPU classifier matches the best transformer GPU model at roughly one-fifth the deployment cost. On D2, CPU classifiers fail via confident miscalibration, producing high-confidence false negatives that bypass escalation entirely. On D3, CPU classifiers outperform transformer GPU models by more than 26 percentage points in F1. Based on these complementary failure modes, we design GuardChain, a three-stage safety pipeline (Regex -> CPU -> GPU) that routes each prompt to the cheapest stage capable of a confident decision. The CPU stage alone resolves 80\% of in-distribution prompts at near-peak accuracy, and the GPU stage recovers the out-of-distribution failures. For practitioners deploying LLM safety at scale, this work provides evidence that GPU-class infrastructure is unnecessary for the majority of traffic.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。