评估教育类大模型辅导系统的提示注入防御,揭示安全、可用性与延迟的权衡关系。
Evaluating Prompt Injection Defenses for Educational LLM Tutors: Security-Usability-Latency Trade-offs
- 设计多层防护流水线,融合模式过滤、结构校验、上下文隔离与会话行为检测。
- 优化后平均延迟约1.5秒,零误报率且攻击绕过率极低。
- 提供可复现评测协议,适合教育机构根据风险偏好选择防护方案。
教育类大模型辅导系统面临核心对齐挑战:既要遵循用户意图,又要维护教学约束与安全策略。本文提出一种针对提示注入防御的评估方法,表明防护机制设计涉及对抗鲁棒性、正常任务可用性与响应延迟之间的明确权衡。我们评估了一种结合确定性模式过滤、结构验证、上下文沙盒与会话级行为检查的领域专用多层防护流水线。在受控保留基准上,该流水线在优化平均延迟下实现低绕过率与低误报率,其操作点优先保障教学可用性(零误报),同时具备可观的攻击抵御能力。我们提供了可复现的基准评测协议,支持在相同条件下进行头对头比较,包含分层自助置信区间、配对McNemar显著性检验、多种子敏感性扫描,以及统一仪器对Prompt Guard与NeMo Guardrails在同一数据集上的直接评估。结果揭示实际权衡:NeMo达到0%绕过率,误报率为16.22%,延迟约1.5秒;Prompt Guard则有38.48%绕过率,误报率3.60%。
原文摘要 · Abstract (English)
Educational LLM tutors face a core AI alignment challenge: they must follow user intent while preserving pedagogical constraints and safety policies. We present an evaluation methodology for prompt-injection defenses in this setting, showing that guardrail design entails explicit trade-offs among adversarial robustness, benign-task usability, and response latency. We evaluate a domain-specific multi-layer safeguard pipeline combining deterministic pattern filters, structural validation, contextual sandboxing, and session-level behavioral checks. On a controlled holdout benchmark, the pipeline reaches low bypass and false positive rates with optimized average latency - an operating point that prioritizes pedagogical usability (zero false positives) while maintaining measurable attack resistance. We provide a reproducible benchmark protocol for head-to-head comparison under identical conditions, including stratified bootstrap confidence intervals, paired McNemar significance tests, multi-seed sensitivity sweeps, and direct evaluation of Prompt Guard and NeMo Guardrails on the same split with unified instrumentation. Results expose operational trade-offs: NeMo reaches 0 percent bypass at 16.22 percent FPR and roughly 1.5s latency, while Prompt Guard yields 38.48 percent bypass with 3.60 percent FPR. The framework supports evidence-based guardrail selection for AI tutoring systems under different institutional risk and usability requirements.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。