直接预测手部网格顶点位置，提升单目3D手部重建精度。
M3DHMR: Monocular 3D Hand Mesh Recovery
- 设计螺旋解码器，动态调整权重提取顶点特征。
- 在FreiHAND数据集上显著优于现有实时方法。
- 适合需要高精度3D手部建模的交互应用。
单目3D手部网格重建因手部自由度高、2D到3D映射模糊及自遮挡而困难。现有方法或效率低，或难以准确预测网格顶点位置。为此，本文提出新框架M3DHMR，直接估计手部网格顶点空间位置。该方法从单张图像提取2D线索用于3D任务，采用包含多个动态螺旋卷积（DSC）层和感兴趣区域（ROI）层的新式螺旋解码器。DSC层根据顶点位置自适应调整权重，在空间与通道维度提取特征；ROI层利用物理先验信息，分区域独立优化网格顶点。在主流数据集FreiHAND上的大量实验表明，M3DHMR显著超越当前最优实时方法。
原文摘要 · Abstract (English)
Monocular 3D hand mesh recovery is challenging due to high degrees of freedom of hands, 2D-to-3D ambiguity and self-occlusion. Most existing methods are either inefficient or less straightforward for predicting the position of 3D mesh vertices. Thus, we propose a new pipeline called Monocular 3D Hand Mesh Recovery (M3DHMR) to directly estimate the positions of hand mesh vertices. M3DHMR provides 2D cues for 3D tasks from a single image and uses a new spiral decoder consist of several Dynamic Spiral Convolution (DSC) Layers and a Region of Interest (ROI) Layer. On the one hand, DSC Layers adaptively adjust the weights based on the vertex positions and extract the vertex features in both spatial and channel dimensions. On the other hand, ROI Layer utilizes the physical information and refines mesh vertices in each predefined hand region separately. Extensive experiments on popular dataset FreiHAND demonstrate that M3DHMR significantly outperforms state-of-the-art real-time methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。