arXiv:2607.16247cs.LGcs.CV2026-07

让智能体主动预判并化解危险,不卡顿也能更安全。

Self-Evolving Just-In-Time Memory for Proactive Embodied Safety

论文配图:Self-Evolving Just-In-Time Memory for Proactive Embodied Safety
图 1 · 摘自论文原文
  • 构建动态记忆系统,持续追踪潜在风险。
  • 测试中自动优化应对策略,安全成功率提升30.3%。
  • 适合需要实时避险的机器人任务场景。

尽管视觉语言模型(VLM)已使具身智能体能够执行复杂的家庭任务,但在闭环交互中仍难以主动应对动态出现的危险。现有安全方法多依赖运行时防护机制来阻止危险动作或导致过度谨慎,严重阻碍任务进展。为打破这一安全与进度的权衡,本文提出自演化即时记忆框架(Self-Evolving Just-In-Time Memory),将具身安全从被动防护转向主动消解。该框架包含:用于在部分可观测下持续追踪安全相关状态的风险充分拓扑信念图(RSG)、基于代理认知的事实记忆以精准预测危险,以及注入程序元技能的经验记忆,以指导可执行且不影响进度的缓解策略。此外,我们设计了自动化测试-验证-写入循环,使智能体可在测试阶段从执行轨迹中持续优化其缓解元技能。在IS-Bench上的实验表明,该框架显著提升了多种VLM骨干模型的安全成功率(如在Qwen3-VL-8B上提升30.3%),实现无需停滞任务进程的主动风险规避。代码已开源。

原文摘要 · Abstract (English)

While Vision-Language Models (VLMs) have empowered embodied agents to execute complex household tasks, they struggle to proactively handle dynamically emerging hazards during closed-loop interactions. Existing safety approaches often rely on runtime guardrails to block unsafe actions or induce excessive caution, which severely stalls task progress instead of actively resolving the underlying risks. To break this safety-progress trade-off, we introduce the Self-Evolving Just-In-Time Memory framework, which reframes embodied safety from progress-stalling guardrails to proactive hazard mitigation. The framework consists of a Risk-Sufficient Topological Belief Graph (RSG) for persistent safety-relevant state tracking under partial observability, an Agency-Grounded Factual Memory for precise hazard anticipation, and an Experience Memory that injects procedural Meta-Skills to guide executable, progress-preserving mitigation. Furthermore, we propose an automated Test-Verify-Write loop, allowing agents to continually refine their mitigation Meta-Skills from execution traces at test time. Experiments on IS-Bench demonstrate that our framework substantially boosts the Safe-Success rate across multiple VLM backbones (e.g., +30.3% on Qwen3-VL-8B), enabling agents to proactively mitigate hazards without stalling task progress. Code is available at https://github.com/DyMessi/JIT-Memory.

具身智能安全增强记忆机制元技能

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。