构建沉浸式救援模拟器,测试智能体在真实场景下的全流程生存搜救能力
RescueBench: Can Embodied Agents Save Lives in the Wild ?

- 设计四阶段流水线:探索、救援、返程、交接,模拟真实救援流程
- 五级难度递进,揭示探索与记忆错误会逐级放大导致任务失败
- 发现现有方法在最高难度下全数失败,探索与空间记忆是主要瓶颈
搜救任务要求具身智能体在多模态不确定性环境下探索陌生场景,执行多阶段交互,并在长时程中保留空间记忆。现有基准通常孤立评估这些能力,难以揭示实际工作流中失败的累积效应。我们提出RescueBench,一个高保真诊断性基准,将搜救建模为四个阶段:多模态探索、目标救援、记忆引导返程和最终交接。通过任务序列组合与阶段级评估,该基准可分析探索与记忆错误在具身救援流程中的传播机制。包含五个渐进难度级别,分别在环境复杂度、线索模糊性和空间层级上变化,并具备自动剧集生成与标注流水线,支持规模化评估与训练。我们评估了七个基线模型、一个理想参考模型及人类玩家,发现在最高难度下所有基线均未能完成全流程任务。阶段级诊断表明,自主探索是主要失败模式,空间记忆是第二个独立瓶颈,说明当前拓扑视觉-语言导航或基于地图的方法尚未解决这些问题。代码已开源于https://github.com/wukui-muc/RescueBench。
原文摘要 · Abstract (English)
Search-and-rescue (SAR) requires embodied agents to explore unfamiliar environments under multimodal uncertainty, perform multi-stage interactions, and retrieve spatial memory over long horizons. Existing benchmarks typically evaluate these capabilities in isolation, leaving unclear how failures compound when they must be composed in realistic workflows. We introduce RescueBench, a photo-realistic diagnostic benchmark that instantiates SAR as a four-stage pipeline: multimodal exploration, target rescue, memory-guided return, and final handoff. By combining sequential task composition with stage-level evaluation, RescueBench enables analysis of how exploration and memory failures propagate through embodied rescue workflows. It contains five progressive difficulty levels that vary in environmental complexity, clue ambiguity, and spatial hierarchy, along with an automatic episode generation and annotation pipeline for scalable evaluation and training. We evaluate seven baselines, an oracle reference, and human players, showing that no baselines complete the full task at the greatest difficulty. Stage-level diagnosis identifies autonomous exploration as the dominant failure mode and spatial memory as a second, independent bottleneck, suggesting that these limitations are not resolved by current topological visual-language navigation or map-based methods. Code is available in https://github.com/wukui-muc/RescueBench
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。