分解手部结构生成更真实的抓握动作,支持刚性与柔性物体。
Human Grasp Generation for Rigid and Deformable Objects with Decomposed VQ-VAE
- 分部件编码手部,提升手指与物体交互精度
- 双阶段解码使抓握类型与位置协同优化,质量提升14.1%
- 新骨架网络与法向量编码,有效模拟物体形变
在计算机图形学与机器人领域,生成真实的人类抓握动作至关重要但极具挑战。现有方法通常将整只手编码并单步估计姿态与位置,难以实现精细的全指-物交互。同时,抓握过程中的物体形变建模仍困难,因需捕捉表面点间复杂关系。为此,我们提出改进的分解式向量量化变分自编码器(DVQ-VAE-2),将手部分解为独立部件分别编码,实现更精确的手-物交互管理。采用双阶段解码策略:先在骨骼约束下预测抓握类型,再确定最优抓握位置,显著提升模型真实感与对未见交互的适应性。引入新型Mesh UFormer作为主干网络,从网格中提取分层结构特征,并设计法向量引导的位置编码以模拟手-物形变。在四个主流基准上,模型抓握质量相对当前最优方法提升约14.1%;与其它主干网络相比,在刚性与柔性物体数据集上分别实现2.23%的指尖接触距离改善和5.86%的质量指数提升。代码与模型已开源于https://github.com/florasion/D-VQVAE。
原文摘要 · Abstract (English)
Generating realistic human grasps is crucial yet challenging for object manipulation in computer graphics and robotics. Current methods often struggle to generate detailed and realistic grasps with full finger-object interaction, as they typically rely on encoding the entire hand and estimating both posture and position in a single step. Additionally, simulating object deformation during grasp generation is still difficult, as modeling such deformation requires capturing the comprehensive relationship among points of the object's surface. To address these limitations, we propose a novel improved Decomposed Vector-Quantized Variational Autoencoder (DVQ-VAE-2), which decomposes the hand into distinct parts and encodes them separately. This part-aware architecture allows for more precise management of hand-object interactions. Furthermore, we introduce a dual-stage decoding strategy that first predicts the grasp type under skeletal constraints and then identifies the optimal grasp position, enhancing both the realism and adaptability of the model to unseen interactions. Furthermore, we introduce a new Mesh UFormer as the backbone network to extract the hierarchical structural representations from the mesh and propose a new normal vector-guided position encoding to simulate the hand-object deformation. In experiments, our model achieves a relative improvement of approximately 14.1% in grasp quality compared to state-of-the-art methods across four widely used benchmarks. Our comparisons with other backbone networks show relative improvements of 2.23% in Hand-object Contact Distance and 5.86% in Quality Index on deformable and rigid object based datasets, respectively. Our source code and model are available at https://github.com/florasion/D-VQVAE.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。