arXiv:2602.19844cs.CRcs.AI2026-02被引 4

LLM应用需系统级监控防范安全风险

LLM-enabled Applications Require System-Level Threat Monitoring

  • 将LLM行为异常视为常态,构建部署后监控机制
  • 现有防御手段无法覆盖运行时安全威胁
  • 适合关注AI系统可靠性的开发者与安全团队

LLM驱动的应用正通过将大语言模型作为核心推理组件,快速重塑软件生态。然而,这种范式转变带来了根本性的可靠性挑战,且因LLM行为的非确定性、学习驱动性和难以验证性,显著扩大了安全攻击面。面对这些不可避免的安全风险,我们主张将其视为预期运营状态而非异常事件,亟需建立专门的事件响应视角。因此,可信部署的主要障碍并非提升模型能力,而是建立系统级威胁监控机制,以在部署后检测并上下文化安全相关异常——这一方面远未被充分探索,超越了测试或护栏式防御。本文呼吁对LLM驱动应用进行系统性、全面的威胁监控,作为可靠运行的前提和专用事件响应框架的基础。

原文摘要 · Abstract (English)

LLM-enabled applications are rapidly reshaping the software ecosystem by using large language models as core reasoning components for complex task execution. This paradigm shift, however, introduces fundamentally new reliability challenges and significantly expands the security attack surface, due to the non-deterministic, learning-driven, and difficult-to-verify nature of LLM behavior. In light of these emerging and unavoidable safety challenges, we argue that such risks should be treated as expected operational conditions rather than exceptional events, necessitating a dedicated incident-response perspective. Consequently, the primary barrier to trustworthy deployment is not further improving model capability but establishing system-level threat monitoring mechanisms that can detect and contextualize security-relevant anomalies after deployment -- an aspect largely underexplored beyond testing or guardrail-based defenses. Accordingly, this position paper advocates systematic and comprehensive monitoring of security threats in LLM-enabled applications as a prerequisite for reliable operation and a foundation for dedicated incident-response frameworks.

LLM安全系统监控可信部署

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。