arXiv:2410.12053cs.CV2024-10被引 1

让3D脑部MRI模型保持旋转一致性,提升解剖细节学习能力。

SOE: SO(3)-Equivariant 3D MRI Encoding

  • 采用SO(3)等变编码,显式建模三维旋转对称性。
  • 在ADNI数据集上预测年龄和阿尔茨海默病准确率优于现有方法。
  • 适用于需要几何敏感性的医学影像分析任务,如脑结构建模。

表示学习在现代模型中愈发重要,尤其在利用脑部解剖结构信息方面。然而,现有针对MRI的模型常忽略或移除平移与旋转等几何信息,导致对几何变换的不变性。我们认为,将这些几何变换知识融入模型可显著提升对脑结构细节的捕捉能力。为此,提出一种新型3D MRI编码方法——SO(3)-等变编码(SOE),强制表示空间对三维空间中所有旋转保持等变性。通过向量神经元实现该性质,使输入图像的旋转操作在嵌入空间中被精确反映。我们在两个公开数据集上预训练SOE,并在ADNI的T1加权脑扫描上评估其在年龄预测与阿尔茨海默病诊断任务中的表现。结果表明,该方法不仅性能更优,且对不同轴向的多种旋转角度均具鲁棒性。代码已开源。

原文摘要 · Abstract (English)

Representation learning has become increasingly important, especially as powerful models have shifted towards learning latent representations before fine-tuning for downstream tasks. This approach is particularly valuable in leveraging the structural information within brain anatomy. However, a common limitation of recent models developed for MRIs is their tendency to ignore or remove geometric information, such as translation and rotation, thereby creating invariance with respect to geometric operations. We contend that incorporating knowledge about these geometric transformations into the model can significantly enhance its ability to learn more detailed anatomical information within brain structures. As a result, we propose a novel method for encoding 3D MRIs that enforces equivariance with respect to all rotations in 3D space, in other words, SO(3)-equivariance (SOE). By explicitly modeling this geometric equivariance in the representation space, we ensure that any rotational operation applied to the input image space is also reflected in the embedding representation space. This approach requires moving beyond traditional representation learning methods, as we need a representation vector space that allows for the application of the same SO(3) operation in that space. To facilitate this, we leverage the concept of vector neurons. The representation space formed by our method captures the brain's structural and anatomical information more effectively. We evaluate SOE pretrained on the structural MRIs of two public data sets with respect to the downstream task of predicting age and diagnosing Alzheimer's Disease from T1-weighted brain scans of the ADNI data set. We demonstrate that our approach not only outperforms other methods but is also robust against various degrees of rotation along different axes. The code is available at https://github.com/shizhehe/SOE-representation-learning.

3D MRI等变表示脑结构分析深度学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。