arXiv:2604.17888cs.RO2026-04

通过分层规划与手-臂解耦,提升复杂空间中的灵巧抓取成功率。

SpaceDex: Generalizable Dexterous Grasping in Tiered Workspaces

论文配图:SpaceDex: Generalizable Dexterous Grasping in Tiered Workspaces
图 1 · 摘自论文原文
  • 高阶用视觉语言模型多视角推理,生成结构化抓取指引。
  • 低阶解耦手臂路径与手部抓握,结合多视角和触觉提升鲁棒性。
  • 实测在4类30+未见物体上成功率达63%,显著优于基线。

在层级式工作空间中,高自由度灵巧手的泛化抓取仍具挑战,因遮挡、狭窄间隙和高度依赖约束远强于开放桌面场景。现有方法多在少遮挡环境下评估,且未显式建模空间约束下机械臂导航与手部动作的差异控制需求。本文提出SpaceDex,一种面向受限三维环境的分层灵巧操作框架。高层采用视觉语言模型(VLM)解析用户意图,跨多视角推理遮挡与高度关系,生成目标边界框用于零样本分割与掩码追踪,提供结构化空间引导而非单视图选择。低层引入手-臂特征分离网络,将机械臂全局轨迹控制与手部几何感知抓握模式选择解耦,减少抓取与抵达目标间的特征干扰。控制器融合多视角感知、指尖触觉传感及少量恢复示范,增强对部分可观测性和非理想接触的鲁棒性。在100次真实世界试验中,涉及4类共30余种未见物体,SpaceDex成功率达63.0%,显著高于强基线的39.0%。结果表明,分层空间规划与手-臂表示解耦可有效提升受限环境下的灵巧抓取性能。

原文摘要 · Abstract (English)

Generalizable grasping with high-degree-of-freedom (DoF) dexterous hands remains challenging in tiered workspaces, where occlusion, narrow clearances, and height-dependent constraints are substantially stronger than in open tabletop scenes. Most existing methods are evaluated in relatively unoccluded settings and typically do not explicitly model the distinct control requirements of arm navigation and hand articulation under spatial constraints. We present SpaceDex, a hierarchical framework for dexterous manipulation in constrained 3D environments. At the high level, a Vision-Language Model (VLM) planner parses user intent, reasons about occlusion and height relations across multiple camera views, and generates target bounding boxes for zero-shot segmentation and mask tracking. This stage provides structured spatial guidance for downstream control instead of relying on single-view target selection. At the low level, we introduce an arm-hand Feature Separation Network that decouples global trajectory control for the arm from geometry-aware grasp mode selection for the hand, reducing feature interference between reaching and grasping objectives. The controller further integrates multi-view perception, fingertip tactile sensing, and a small set of recovery demonstrations to improve robustness to partial observability and off-nominal contacts. In 100 real-world trials involving over 30 unseen objects across four categories, SpaceDex achieves a 63.0\% success rate, compared with 39.0\% for a strong tabletop baseline. These results indicate that combining hierarchical spatial planning with arm-hand representation decoupling improves dexterous grasping performance in spatially constrained environments.

灵巧抓取分层控制多模态感知

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。