轻量级Transformer架构可高效检测大模型恶意意图,适合生产部署。
JavelinGuard: Low-Cost Transformer Architectures for LLM Security
- 设计五种渐进式轻量Transformer模型,参数低至4亿
- 在九个对抗数据集上达到高精度,推理速度优于标准CPU
- 适合对效率与安全平衡有要求的工业级应用
我们提出JavelinGuard,一套专为生产环境设计的低成本、高性能模型架构,用于检测大语言模型(LLM)交互中的恶意意图。基于紧凑型Transformer架构(如ModernBERT),仅需约4亿参数即可实现高精度分类,并在标准CPU上实现快速推理。系统评估了五种逐步复杂的架构:Sharanga(基础Transformer)、Mahendra(增强注意力池化)、Vaishnava和Ashwina(混合神经集成)、Raudra(多任务框架,含专用损失函数)。在包括NotInject、BIPIA、Garak、ImprovedLLM、ToxicChat、WildGuard及新提出的JavelinBench在内的九个对抗数据集上进行严格测试,后者专门针对边界案例和难负样本。相比主流开源护栏模型及gpt-4o等大解码器模型,本系列架构在准确率与延迟间展现更优性价比。结果显示,尽管Raudra整体表现最强,但各模型在速度、可解释性与资源消耗间各有权衡,为实际应用提供选型依据。
原文摘要 · Abstract (English)
We present JavelinGuard, a suite of low-cost, high-performance model architectures designed for detecting malicious intent in Large Language Model (LLM) interactions, optimized specifically for production deployment. Recent advances in transformer architectures, including compact BERT(Devlin et al. 2019) variants (e.g., ModernBERT (Warner et al. 2024)), allow us to build highly accurate classifiers with as few as approximately 400M parameters that achieve rapid inference speeds even on standard CPU hardware. We systematically explore five progressively sophisticated transformer-based architectures: Sharanga (baseline transformer classifier), Mahendra (enhanced attention-weighted pooling with deeper heads), Vaishnava and Ashwina (hybrid neural ensemble architectures), and Raudra (an advanced multi-task framework with specialized loss functions). Our models are rigorously benchmarked across nine diverse adversarial datasets, including popular sets like the NotInject series, BIPIA, Garak, ImprovedLLM, ToxicChat, WildGuard, and our newly introduced JavelinBench, specifically crafted to test generalization on challenging borderline and hard-negative cases. Additionally, we compare our architectures against leading open-source guardrail models as well as large decoder-only LLMs such as gpt-4o, demonstrating superior cost-performance trade-offs in terms of accuracy, and latency. Our findings reveal that while Raudra's multi-task design offers the most robust performance overall, each architecture presents unique trade-offs in speed, interpretability, and resource requirements, guiding practitioners in selecting the optimal balance of complexity and efficiency for real-world LLM security applications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。