arXiv:2608.04289cs.AIcs.CL2026-08

提出安全承诺机制,让智能体在记忆不确定时避免提前行动

SafeCommit: Certifying When Memory-Grounded Agents May Safely Act

  • 构建可校准的潜在世界集合,评估记忆可信度
  • 仅当所有可能世界均安全时才执行动作,否则探测或回退
  • 适合高风险场景下的自主决策系统,如医疗、金融

长时序智能体越来越多地依赖持久化记忆和工具执行具有外部影响的动作。一个核心失败模式是过早承诺:智能体在未确认记忆是否过期、冲突、不完整或损坏的情况下就采取行动。我们将其形式化为记忆不确定性下的安全承诺问题,并提出SafeCommit——位于智能体推理与外部执行之间的风险可控层。该层从记忆、观测、工具输出、溯源信息及策略约束中构建校准后的可能潜在世界集合。仅当符合性动作证书表明该动作在所有保留世界中均安全时,才允许执行;否则选择低影响探测动作以排除阻碍认证的世界,或返回保守回退。在世界覆盖校准下,不安全认证动作的概率不超过目标水平α;在世界生成不完美时,该界分离了校准误差与表征误差。一个无依赖的受控模拟器展示了安全-效用权衡,并可通过单条命令复现全部结果。目标是提供一种具体方法,不仅决定智能体该做什么,更判断现有证据是否足以安全执行。

原文摘要 · Abstract (English)

Long-horizon agents increasingly use persistent memory and tools to take actions with external side effects. A central failure mode is premature commitment: an agent acts before resolving whether its memory grounding is stale, conflicting, incomplete, or corrupted. We formalize this problem as safe commitment under memory uncertainty and introduce SafeCommit, a risk controlled layer between agent reasoning and external execution. The layer constructs a calibrated set of plausible latent worlds from memory, observations, tool outputs, provenance, and policy constraints. It permits a side effectful action only when a conformal action certificate shows that the action is safe in every retained world. Otherwise, it selects a low-side-effect probe that targets the worlds blocking certification, or returns a conservative fallback. Under calibrated world coverage, the probability of an unsafe certified commit is at most the target level α; with imperfect world proposal, the bound separates calibration and representation error. A dependency-free controlled simulator illustrates the safety-utility tradeoff and reproduces all reported results with one command. The goal is to offer a concrete approach for deciding not only what an agent should do, but when the available evidence is sufficient to safely do it.

智能体安全记忆验证决策可靠性

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。