用单张照片精准估脊柱形态,避免拍X光的辐射风险。
A Dual-Feature Extractor Framework for Accurate Back Depth and Spine Morphology Estimation from Monocular RGB Images
- 双分支网络提取局部与全局特征,融合深度与表面信息
- 深度估计准确率超97.5%,脊柱曲线重建精度达97%
- 适合无辐射筛查脊柱侧弯,尤其适用于偏远地区
脊柱侧弯是一种常见病症,其中青少年特发性脊柱侧弯(AIS)最为普遍。目前主要依赖X光检查,但存在辐射暴露和偏远地区可及性差的问题。为此,本研究提出一种新方法,通过单目RGB图像估算裸背深度信息,并结合表面特征进行脊柱形态分析。为精确捕捉背部微小深度变化,设计了网格感知多尺度自适应网络(GAMA-Net),采用双编码器分别提取局部块级与全局特征,通过基于块的混合注意力模块实现交互,解码器中使用自适应多尺度特征融合模块动态融合信息。深度估计在三个评估指标上分别达到近78.2%、93.6%和97.5%。进一步将表面与深度信息整合用于脊柱形态估计,显著提升脊柱曲线生成精度,最高达97%。
原文摘要 · Abstract (English)
Scoliosis is a prevalent condition that impacts both physical health and appearance, with adolescent idiopathic scoliosis (AIS) being the most common form. Currently, the main AIS assessment tool, X-rays, poses significant limitations, including radiation exposure and limited accessibility in poor and remote areas. To address this problem, the current solutions are using RGB images to analyze spine morphology. However, RGB images are highly susceptible to environmental factors, such as lighting conditions, compromising model stability and generalizability. Therefore, in this study, we propose a novel pipeline to accurately estimate the depth information of the unclothed back, compensating for the limitations of 2D information, and then estimate spine morphology by integrating both depth and surface information. To capture the subtle depth variations of the back surface with precision, we design an adaptive multiscale feature learning network named Grid-Aware Multiscale Adaptive Network (GAMA-Net). This model uses dual encoders to extract both patch-level and global features, which are then interacted by the Patch-Based Hybrid Attention (PBHA) module. The Adaptive Multiscale Feature Fusion (AMFF) module is used to dynamically fuse information in the decoder. As a result, our depth estimation model achieves remarkable accuracy across three different evaluation metrics, with scores of nearly 78.2%, 93.6%, and 97.5%, respectively. To further validate the effectiveness of the predicted depth, we integrate both surface and depth information for spine morphology estimation. This integrated approach enhances the accuracy of spine curve generation, achieving an impressive performance of up to 97%.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。