DreamGuard通过风险感知世界模型，提前识别长时程潜在危险，提升LLM代理安全性。
DreamGuard: Efficient Runtime Guardrail for LLM Agents via Risk-Aware World Model
- 构建风险感知世界模型，跟踪轨迹隐状态预测未来风险
- 在4个基准上优于其他防护方案，安全-效率平衡最优
- 适合需长期安全运行的LLM代理应用，如自动化工具调用
随着大语言模型（LLM）代理越来越多地调用外部工具并与真实系统交互，不安全行为可能对外部状态、用户数据和下游服务造成不可逆后果。现有运行时防护机制多为被动响应：仅评估当前动作的表面安全性，缺乏对风险随轨迹演化的显式建模。这一局限导致长时程风险难以察觉——看似无害的动作可能逐步引导代理进入危险状态。为此，我们提出DreamGuard，一种基于风险感知世界模型的主动防护机制。该世界模型在轨迹上维护紧凑的递归隐状态，并从未来隐状态预测中推导即时危害与前缀风险证据。随后将多时域信号融合，生成执行前的干预决策。在四个基准及线上防护评估中，DreamGuard超越通用、被动与主动基线，在所评估防护方案中实现最佳安全-效用权衡，且每调用平均端到端延迟仅为25毫秒。
原文摘要 · Abstract (English)
As large language model (LLM) agents increasingly invoke external tools and interact with real-world systems, unsafe actions may cause irreversible consequences on external states, user data, and downstream services. Recent runtime guardrails mitigate such risks by checking proposed actions before execution, but many remain reactive: they primarily assess the apparent safety of the current action, lacking an explicit model of how risk evolves across the trajectory. This limitation creates a critical blind spot for long-horizon risks, where individually benign-looking actions can gradually drift the agent toward hazardous states. In response, we propose DreamGuard, a proactive guardrail for LLM agents built around a risk-aware world model. The world model maintains a compact recurrent latent state over the trajectory and predicts future latent states from which DreamGuard derives immediate-hazard and prefix-risk evidence. It then fuses these multi-horizon signals into intervention decisions before execution. Experiments across four benchmarks and an online guardrail evaluation show that DreamGuard outperforms generic, reactive, and proactive guardrail baselines, achieves the best safety-utility trade-off among evaluated guardrails, and maintains an average end-to-end latency of 25 ms per call.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。