用几何变形自动发现机器人抓取的脆弱点,提升系统鲁棒性。
Geometric Red-Teaming for Robotic Manipulation
- 通过几何变形生成能引发崩溃的形状,自动探测策略缺陷。
- 在插入、铰接和抓取任务中,使成功率从90%降至22.5%。
- 可用于针对性优化策略,实测恢复成功率至90%。
机器人操作的标准评估通常在精心挑选的分布内测试集上进行,难以揭示系统在合理变化下的失效情况。我们提出几何红队(GRT),一种通过对象中心的几何扰动探测鲁棒性的红队框架,可自动生成结构合法、用户约束的网格变形(CrashShapes),触发预训练操作策略的灾难性失败。该方法结合基于雅可比场的变形模型与无梯度的仿真闭环优化策略。在插入、铰接和抓取任务中,GRT持续发现导致策略性能崩溃的形变,揭示了静态基准遗漏的脆弱失效模式。通过任务级策略回放与约束感知形状探索相结合,我们旨在构建通用的、以对象为中心的鲁棒性评估框架。此外,我们发现对单个CrashShape进行微调(称为蓝队)可使该形状上的任务成功率提升最高达60个百分点,同时保持原物体性能,证明了红队几何对策略精炼的有效性。最后,我们在真实机械臂上验证结果:模拟的CrashShapes使任务成功率从90%降至最低22.5%,而蓝队修复后恢复至最高90%,与仿真结果高度一致。视频与代码见项目主页:https://georedteam.github.io/。
原文摘要 · Abstract (English)
Standard evaluation protocols in robotic manipulation typically assess policy performance over curated, in-distribution test sets, offering limited insight into how systems fail under plausible variation. We introduce Geometric Red-Teaming (GRT), a red-teaming framework that probes robustness through object-centric geometric perturbations, automatically generating CrashShapes -- structurally valid, user-constrained mesh deformations that trigger catastrophic failures in pre-trained manipulation policies. The method integrates a Jacobian field-based deformation model with a gradient-free, simulator-in-the-loop optimization strategy. Across insertion, articulation, and grasping tasks, GRT consistently discovers deformations that collapse policy performance, revealing brittle failure modes missed by static benchmarks. By combining task-level policy rollouts with constraint-aware shape exploration, we aim to build a general purpose framework for structured, object-centric robustness evaluation in robotic manipulation. We additionally show that fine-tuning on individual CrashShapes, a process we refer to as blue-teaming, improves task success by up to 60 percentage points on those shapes, while preserving performance on the original object, demonstrating the utility of red-teamed geometries for targeted policy refinement. Finally, we validate both red-teaming and blue-teaming results with a real robotic arm, observing that simulated CrashShapes reduce task success from 90% to as low as 22.5%, and that blue-teaming recovers performance to up to 90% on the corresponding real-world geometry -- closely matching simulation outcomes. Videos and code can be found on our project website: https://georedteam.github.io/ .
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。