通过合并冗余层与令牌，加速3D人体网格恢复，性能微升。
FastHMR: Accelerating Human Mesh Recovery via Token and Layer Merging with Diffusion Decoding
- 按误差约束合并低影响的Transformer层，减少计算量。
- 用掩码引导合并背景令牌，保留关键特征。
- 引入扩散解码器融合时序信息，提升恢复精度。
基于Transformer的3D人体网格恢复（HMR）模型虽性能优异，但因深层架构和冗余令牌导致计算开销大。本文提出两种专用于HMR的合并策略：误差约束层合并（ECLM）与掩码引导令牌合并（Mask-ToMe）。ECLM选择对平均关节位置误差（MPJPE）影响最小的层进行合并，而Mask-ToMe聚焦于合并对最终预测贡献较小的背景令牌。为缓解合并带来的性能下降，提出一种基于扩散的解码器，融合时序上下文并利用大规模动作捕捉数据学习的姿态先验。在多个基准测试中，方法实现最高2.3倍加速，同时性能略优于基线。
原文摘要 · Abstract (English)
Recent transformer-based models for 3D Human Mesh Recovery (HMR) have achieved strong performance but often suffer from high computational cost and complexity due to deep transformer architectures and redundant tokens. In this paper, we introduce two HMR-specific merging strategies: Error-Constrained Layer Merging (ECLM) and Mask-guided Token Merging (Mask-ToMe). ECLM selectively merges transformer layers that have minimal impact on the Mean Per Joint Position Error (MPJPE), while Mask-ToMe focuses on merging background tokens that contribute little to the final prediction. To further address the potential performance drop caused by merging, we propose a diffusion-based decoder that incorporates temporal context and leverages pose priors learned from large-scale motion capture datasets. Experiments across multiple benchmarks demonstrate that our method achieves up to 2.3x speed-up while slightly improving performance over the baseline.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。