提出新评估框架,量化机器人系统在扰动下的恢复与稳定性。
Resilience Matters for Embodied Agents System: New Metrics, Systematic Evaluation, and Optimization

- 引入反弹、稳定性和渐进扩展三维度韧性指标,揭示执行过程细节。
- 实验显示成功任务间恢复成本差达25.2,且存在任务家族退化现象。
- 指标指导优化可降低恢复成本,提升系统韧性,适合部署场景定制。
具身智能体系统(EAS)在开放物理世界中部署日益增多,其可靠性直接决定部署质量与人机信任。现有评估依赖成功率或安全评分等结果导向指标,将多样执行轨迹简化为粗略分数,掩盖了智能体行为背后的动态过程。为此,我们提出“韧性”这一关键属性——即系统在扰动和迭代更新下恢复、稳定与延展的能力,并基于韧性工程思想构建首个适用于任意EAS的综合性韧性评估框架。该框架定义了反弹、稳定性和渐进扩展三个核心指标,实现对具身任务执行过程的精细刻画。通过构建韧性评估层,将执行过程转化为可诊断、可优化的评估结果。在400个家庭任务、10个EAS上的实验表明,结果指标掩盖了过程差异:成功任务间的恢复成本差达ΔC_rec=25.2,且存在不稳定性上升与任务家族性能退化。基于指标的优化显著降低了恢复成本,提升了稳定性和渐进扩展完成率,验证了评估的诊断价值。研究发现韧性特征间存在权衡,建议根据具体部署需求配置韧性系统。
原文摘要 · Abstract (English)
Embodied Agents System (EAS) are increasingly deployed in open-world physical domains, where reliability directly dictates deployment quality and human-agent trust. However, existing evaluations rely on outcome-centric metrics as success rate or safety scores that collapse diverse execution trajectories into coarse scores, obscuring the dynamic processes underlying agent behavior. Therefore, they ignore a critical property of EAS -- which we define as the Resilience -- that reflects how EASs recover, stabilize, and extend under perturbations and across iterative updates. The lack of resilience is particularly critical in open-world environments due to continuous unexpected disruptions, thus directly affecting the quality of EAS deployment. To address this problem, we gain insight from the resilience-engineering concepts to EAS groundings and propose a novel resilience evaluation framework that can be flexibly applied to any EAS. Specifically, we define the first comprehensive resilience metrics suite for EASs system that exposes Rebound, Stability, and Graceful Extensibility across embodied tasks execution, providing a practical grounding for EAS resilience analysis. We further implement the resilience evaluation layer that transforms execution process into assessments for diagnosis and optimization. Across 400 household tasks with 10 EAS, we reveal the process-level distinction hidden by outcome metrics, including recovery cost differences among successful episodes ($ΔC_{rec}=25.2$), increased instability and task-family degradation. Metrics-guided optimizations reduce recovery cost and increase stability, graceful extensibility completion, showing the diagnostic effect of resilience evaluation. Our results reveal a trade-off among resilience characteristics, suggesting that a resilient EAS construction should be configured according to deployment-specific requirements.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。