提出新基准SIMMER,检测大模型规划中的隐蔽失败问题。
SIMMER: Benchmarking Latent Failures in LLM Executable Planning with a World Model

- 构建厨房场景符号世界模型,包含46800种语义合理交互。
- 六款大模型最多仅17%计划无错误,56%存在致命潜伏缺陷。
- 通过反事实预演推理,可减少72%潜伏失败,适合安全敏感应用研究。
大型语言模型(LLMs)正被广泛用于家庭环境中自主代理的规划。现有评估基准仅关注计划是否成功执行,却忽略了关键的潜在失败类型:潜伏失败。与执行时立即触发反馈并可及时修正的即时失败不同,潜伏失败不会立刻中断计划执行,但会悄然损害目标达成,严重时造成不可逆伤害。为填补这一空白,我们提出SIMMER,一个基于人工标注符号世界模型的基准,专门评估LLM规划中的潜伏失败。该世界模型以厨房场景为基础,包含77个动作、262种独特物体和约46,800种语义合理的交互,源自真实烹饪脚本。通过状态机执行器验证计划,可检测即时先决条件违规、潜伏风险及不可逆失败。在六款大模型上的实验表明,即使前沿模型也仅有最多17%的计划无错误;高达56%的计划包含潜伏失败,多数导致不可逆后果。进一步实验证明,通过反事实预见模拟进行显式状态推理,可将潜伏失败降低72%,不可逆情况减少75%,为构建更鲁棒的LLM规划器指明方向。
原文摘要 · Abstract (English)
Large language models (LLMs) are increasingly deployed as planners for autonomous agents in household environments. While existing benchmarks evaluate whether LLM-generated plans execute successfully, they overlook a critical type of failure: latent failures. Unlike immediate failures that trigger instant feedback at execution time and enable timely correction, latent failures do not immediately halt plan execution but silently compromise goal achievement. In severe cases, they cause irreversible harm. To address this gap, we introduce SIMMER, a benchmark for evaluating latent failures in LLM planning through a human-curated symbolic world model grounded in the kitchen domain. SIMMER defines a world model comprising 77 actions, 262 unique objects, and approximately 46,800 possible interactions that are semantically realistic, derived from real-world cooking scripts. It then leverages a state machine executor that validates plans against the world model and detects immediate precondition violations, latent hazards, and irreversible failures. Experiments across six LLMs show that even frontier models achieve at most 17% error-free plans. Moreover, up to 56% of plans contain latent failures, the majority of which lead to irreversible consequences. We further demonstrate that explicit state reasoning via counterfactual foresight simulation can reduce latent failures by up to 72% and irreversible cases by up to 75%, suggesting a promising direction for more robust LLM planners.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。