仅用一张RGB图实时生成高精度人体骨骼,速度快超800倍。
EA-RAS: Towards Efficient and Accurate End-to-End Reconstruction of Anatomical Skeleton
- 单阶段端到端架构,仅需单张图片输入,轻量可插拔。
- 速度比现有方法快800倍以上,后处理可提升精度50%以上。
- 适合实时应用,如人机交互与生物教育,兼顾准确与效率。
高效、准确且低成本地估计人体骨骼信息对生物学教育、人机交互等应用至关重要。然而,当前基于2D-3D关节点的简化骨骼模型在解剖保真度上不足,限制了其应用;而更复杂的解剖模型虽精确,却依赖多阶段处理和额外数据(如皮肤网格),难以实现实时计算。为此,我们提出EA-RAS(高效精准端到端解剖骨骼重建),一种单阶段、轻量、即插即用的解剖骨骼估计器,仅需单张RGB图像即可实现任意姿态下实时、高保真的解剖学真实骨骼重建。此外,EA-RAS显式估计传统人体网格模型,不仅增强功能,还通过将外部皮肤特征融入内部骨骼建模过程,提升表现。本文还设计了一种渐进式训练策略,并结合改进优化过程,使网络仅用少量皮肤数据即可获得初始权重,并实现骨骼重建的自监督。此外,提供可选轻量级后处理优化策略,适用于对精度优先但不苛求实时性的场景。实验表明,该回归方法比现有方法快800倍以上,满足实时需求;后处理策略可使重建精度提升超过50%,并带来7倍以上的速度提升。
原文摘要 · Abstract (English)
Efficient, accurate and low-cost estimation of human skeletal information is crucial for a range of applications such as biology education and human-computer interaction. However, current simple skeleton models, which are typically based on 2D-3D joint points, fall short in terms of anatomical fidelity, restricting their utility in fields. On the other hand, more complex models while anatomically precise, are hindered by sophisticate multi-stage processing and the need for extra data like skin meshes, making them unsuitable for real-time applications. To this end, we propose the EA-RAS (Towards Efficient and Accurate End-to-End Reconstruction of Anatomical Skeleton), a single-stage, lightweight, and plug-and-play anatomical skeleton estimator that can provide real-time, accurate anatomically realistic skeletons with arbitrary pose using only a single RGB image input. Additionally, EA-RAS estimates the conventional human-mesh model explicitly, which not only enhances the functionality but also leverages the outside skin information by integrating features into the inside skeleton modeling process. In this work, we also develop a progressive training strategy and integrated it with an enhanced optimization process, enabling the network to obtain initial weights using only a small skin dataset and achieve self-supervision in skeleton reconstruction. Besides, we also provide an optional lightweight post-processing optimization strategy to further improve accuracy for scenarios that prioritize precision over real-time processing. The experiments demonstrated that our regression method is over 800 times faster than existing methods, meeting real-time requirements. Additionally, the post-processing optimization strategy provided can enhance reconstruction accuracy by over 50% and achieve a speed increase of more than 7 times.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。