arXiv:2410.07995cs.CV2024-10ECCV被引 1

根据物体接触区域生成多样自然的手部抓握姿势。

RegionGrasp: A Novel Task for Contact Region Controllable Hand Grasp Generation

论文配图:RegionGrasp: A Novel Task for Contact Region Controllable Hand Grasp Generation
图 1 · 摘自论文原文
  • 通过条件编码器捕捉接触区域信息,结合掩码重建预训练提升几何感知。
  • 引入交互编码网络,融合手部与物体特征实现精准抓握建模。
  • 适合机器人抓取、虚拟仿真等需要可控手部动作的场景。

能否让机器在给定3D物体特定接触区域的情况下,自动生成多个不同且自然的手部抓握姿势?为此,我们提出了一项新任务——接触区域可控手部抓握生成(RegionGrasp)。输入为一个3D物体及其指定的接触表面区域,目标是生成一组合理的手部抓握姿态,其中拇指指尖需落在该接触区域内。为此,我们提出了RegionGrasp-CVAE模型,包含两个核心部分:首先,为实现接触区域感知,设计了以Transformer为骨干的条件编码器ConditionNet,其中对象编码器O-Enc采用点云表面片段随机掩码并重建的预训练策略,以增强对物体表面几何信息的捕捉能力;其次,为实现交互感知,引入HOINet,通过几何感知的多头交叉注意力机制,将高层手部特征与嵌入式物体特征进行纠缠编码,以建模精细的交互关系。实验结果表明,该方法在定性和定量评估上均优于当前最先进方法。

原文摘要 · Abstract (English)

Can machine automatically generate multiple distinct and natural hand grasps, given specific contact region of an object in 3D? This motivates us to consider a novel task of \textit{Region Controllable Hand Grasp Generation (RegionGrasp)}, as follows: given as input a 3D object, together with its specific surface area selected as the intended contact region, to generate a diverse set of plausible hand grasps of the object, where the thumb finger tip touches the object surface on the contact region. To address this task, RegionGrasp-CVAE is proposed, which consists of two main parts. First, to enable contact region-awareness, we propose ConditionNet as the condition encoder that includes in it a transformer-backboned object encoder, O-Enc; a pretraining strategy is adopted by O-Enc, where the point patches of object surface are randomly masked off and subsequently restored, to further capture surface geometric information of the object. Second, to realize interaction awareness, HOINet is introduced to encode hand-object interaction features by entangling high-level hand features with embedded object features through geometric-aware multi-head cross attention. Empirical evaluations demonstrate the effectiveness of our approach qualitatively and quantitatively where it is shown to compare favorably with respect to the state of the art methods.

手部抓握3D生成条件生成交互建模

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。