arXiv:2603.16444cs.CV2026-03ICCV

用知识蒸馏让手部三维重建模型变快又轻,适合手机等设备实时运行。

Fast-HaMeR: Boosting Hand Mesh Reconstruction using Knowledge Distillation

  • 用轻量网络替换原模型主干,配合输出与特征级蒸馏加速推理
  • 模型体积减至35%,速度提升1.5倍,精度仅降0.4mm
  • 成果可直接用于VR/AR、机器人等低功耗场景

快速准确的3D手部重建对VR/AR、人机交互、机器人和医疗等实时应用至关重要。现有先进方法多依赖重型模型,难以在头显、手机和嵌入式系统等资源受限设备上部署。本文研究如何结合轻量神经网络与知识蒸馏,加速复杂3D手部重建模型,使其更快速、更轻量,同时保持相近的重建精度。尽管该方法适用于多种重建框架,我们重点优化当前精度领先的HaMeR模型。将原ViT-H主干替换为MobileNet、MobileViT、ConvNeXt和ResNet等轻量结构,并测试输出级、特征级及二者混合的三种蒸馏策略。实验表明,使用仅原模型35%大小的轻量主干,可实现1.5倍的推理速度提升,且精度损失仅0.4mm。具体而言,输出级蒸馏显著提升学生模型性能,而特征级蒸馏对高容量学生模型更有效。研究成果为低功耗设备上的高效应用铺平道路。代码与模型已公开于https://github.com/hunainahmedj/Fast-HaMeR。

原文摘要 · Abstract (English)

Fast and accurate 3D hand reconstruction is essential for real-time applications in VR/AR, human-computer interaction, robotics, and healthcare. Most state-of-the-art methods rely on heavy models, limiting their use on resource-constrained devices like headsets, smartphones, and embedded systems. In this paper, we investigate how the use of lightweight neural networks, combined with Knowledge Distillation, can accelerate complex 3D hand reconstruction models by making them faster and lighter, while maintaining comparable reconstruction accuracy. While our approach is suited for various hand reconstruction frameworks, we focus primarily on boosting the HaMeR model, currently the leading method in terms of reconstruction accuracy. We replace its original ViT-H backbone with lighter alternatives, including MobileNet, MobileViT, ConvNeXt, and ResNet, and evaluate three knowledge distillation strategies: output-level, feature-level, and a hybrid of both. Our experiments show that using lightweight backbones that are only 35% the size of the original achieves 1.5x faster inference speed while preserving similar performance quality with only a minimal accuracy difference of 0.4mm. More specifically, we show how output-level distillation notably improves student performance, while feature-level distillation proves more effective for higher-capacity students. Overall, the findings pave the way for efficient real-world applications on low-power devices. The code and models are publicly available under https://github.com/hunainahmedj/Fast-HaMeR.

手部重建知识蒸馏轻量化实时应用

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。