多视角注意力融合实现实时高精度3D手部姿态与形状估计
HandMvNet: Real-Time 3D Hand Pose Estimation Using Multi-View Cross-Attention Fusion
- 通过多视角交叉注意力机制融合不同视角特征
- 无需相机参数即可准确恢复绝对3D手部姿态和形状
- 推理速度快，适合实时应用，性能超越现有方法
本文提出HandMvNet，一种首个可实时运行的多视角图像3D手部动作与形状估计方法。与以往单目方法因尺度-深度模糊导致精度受限不同，本方法通过多视角注意力融合机制，实现了稳定且精确的绝对3D手部姿态与形状重建。相比传统多视角方法需输入相机参数来学习3D几何，HandMvNet无需依赖相机参数即可完成建模。在保持实时推理速度的同时，其性能在公开数据集上优于当前主流方法，在相同设置下定性与定量评估均表现更优。代码已开源：github.com/pyxploiter/handmvnet。
原文摘要 · Abstract (English)
In this work, we present HandMvNet, one of the first real-time method designed to estimate 3D hand motion and shape from multi-view camera images. Unlike previous monocular approaches, which suffer from scale-depth ambiguities, our method ensures consistent and accurate absolute hand poses and shapes. This is achieved through a multi-view attention-fusion mechanism that effectively integrates features from multiple viewpoints. In contrast to previous multi-view methods, our approach eliminates the need for camera parameters as input to learn 3D geometry. HandMvNet also achieves a substantial reduction in inference time while delivering competitive results compared to the state-of-the-art methods, making it suitable for real-time applications. Evaluated on publicly available datasets, HandMvNet qualitatively and quantitatively outperforms previous methods under identical settings. Code is available at github.com/pyxploiter/handmvnet.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。