arXiv:2409.18701eess.IVcs.CV2024-09被引 1

用3D结构重建提升全景牙片分析精度,解决2D信息丢失问题

3DPX: Single Panoramic X-ray Analysis Guided by 3D Oral Structure Reconstruction

  • 分层渐进重建3D口腔结构,融合多层语义信息增强准确性
  • 通过双向特征对齐与对比引导,提升2D与3D图像协同分析能力
  • 在464例数据上超越现有方法,适用于牙科病变检测与分类

全景牙片(PX)因普及性强和成本低,在牙科中广泛应用。但作为三维结构的二维投影,其存在解剖信息丢失问题,诊断能力弱于三维成像。已有2D到3D重建方法尝试从2D PX中复原缺失的3D信息,但面临两大挑战:一是2D图像深度推断准确率有限;二是如何有效联合分析2D与合成3D图像,以最大化二者协同效应并最小化合成误差。本文提出3DPX——一种基于2D-to-3D重建的全景牙片分析新方法,包含两个核心组件:(i) 一种新型渐进式重建网络,通过金字塔多层级中间重建结果注入知识,并引入多层感知机增强语义理解;(ii) 一种对比引导的双向多模态对齐模块,利用重建3D图像作为解剖指导,实现2D PX分类与分割任务中的特征对齐。该模块通过双向投影提升2D-3D协同性,同时以对比学习减少合成图像带来的误差影响。在两个口腔数据集(共464例)上的实验表明,3DPX在2D-to-3D重建、分类及病灶分割等多项任务中均优于现有最先进方法。

原文摘要 · Abstract (English)

Panoramic X-ray (PX) is a prevalent modality in dentistry practice owing to its wide availability and low cost. However, as a 2D projection of a 3D structure, PX suffers from anatomical information loss and PX diagnosis is limited compared to that with 3D imaging modalities. 2D-to-3D reconstruction methods have been explored for the ability to synthesize the absent 3D anatomical information from 2D PX for use in PX image analysis. However, there are challenges in leveraging such 3D synthesized reconstructions. First, inferring 3D depth from 2D images remains a challenging task with limited accuracy. The second challenge is the joint analysis of 2D PX with its 3D synthesized counterpart, with the aim to maximize the 2D-3D synergy while minimizing the errors arising from the synthesized image. In this study, we propose a new method termed 3DPX - PX image analysis guided by 2D-to-3D reconstruction, to overcome these challenges. 3DPX consists of (i) a novel progressive reconstruction network to improve 2D-to-3D reconstruction and, (ii) a contrastive-guided bidirectional multimodality alignment module for 3D-guided 2D PX classification and segmentation tasks. The reconstruction network progressively reconstructs 3D images with knowledge imposed on the intermediate reconstructions at multiple pyramid levels and incorporates Multilayer Perceptrons to improve semantic understanding. The downstream networks leverage the reconstructed images as 3D anatomical guidance to the PX analysis through feature alignment, which increases the 2D-3D synergy with bidirectional feature projection and decease the impact of potential errors with contrastive guidance. Extensive experiments on two oral datasets involving 464 studies demonstrate that 3DPX outperforms the state-of-the-art methods in various tasks including 2D-to-3D reconstruction, PX classification and lesion segmentation.

口腔影像2D到3D重建多模态对齐牙科诊断

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。