让智能体主动识别动作风险，避免误判或漏判危险。
EMBGuard: Constructing Hazard-Aware Guardrails for Safe Planning in Embodied Agents
- 分离安全判断与决策策略，通过视觉+动作判断是否危险
- 在329个真实场景中表现优于大模型，假阳性率显著降低
- 适合需要高安全性的机器人规划，如家庭服务或工业巡检
部署于真实环境的多模态大模型驱动智能体面临物理风险。现有方法缺乏显式识别危害和推理动作相关风险的机制，导致智能体要么遗漏危险交互，要么过度误判风险。为此，我们提出 EMBGuard，首个基于多模态大模型的安全护栏，可将物理风险推理与智能体策略解耦。EMBGuard 通过评估（视觉观测，动作）对，识别潜在危险配置，并提供自然语言解释。我们同时构建了包含15.1K个动作条件样本的 EMBHazard 训练数据集，以及涵盖七类物理风险的329个手动标注的真实场景基准 EMBGuardTest。通过组合变化生成多样化风险与安全场景。尽管模型规模仅2B、4B，EMBGuard 性能媲美专有大模型（如 GPT-5.1、Gemini-2.5-Pro），且显著降低阻碍实时部署的假阳性率。代码、数据与模型已开源。
原文摘要 · Abstract (English)
MLLM-powered embodied agents deployed in real-world environments encounter physical hazards. However, existing approaches lack explicit mechanisms for identifying hazards and reasoning about action-conditioned risks, leading agents to either miss risky interactions or over-identify risks. To address this, we propose EMBGuard, the first MLLM-based safety guardrail for embodied agents designed to decouple physical risk reasoning from agent policy. By evaluating a (visual observation, action) pair, EMBGuard identifies hazardous configurations and provides natural language explanations of potential risks. Alongside EMBGuard, we contribute EMBHazard, a training dataset of 15.1K action-conditioned pairs, and EMBGuardTest, a benchmark of 329 manually curated real-world scenarios spanning seven physical risk categories. Through compositional variation of hazards and actions, we generate diverse risky and benign scenarios that agents may encounter during planning. Despite its compact size (2B, 4B), EMBGuard achieves performance competitive with proprietary MLLMs (e.g., GPT-5.1, Gemini-2.5-Pro) while significantly reducing the false-positive rates that hinder real-time deployment. We make the code, data, and models publicly available at https://github.com/dongwxxkchoi/EMBGuard
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。