用2D图像知识提升3D神经元重建精度
DINeuro: Distilling Knowledge from 2D Natural Images via Deformable Tubular Transferring Strategy for 3D Neuron Reconstruction
- 将2D图像预训练知识通过可变形管状迁移适配到3D神经元结构
- 在BigNeuron数据集上提升平均Dice 4.53%，Hausdorff距离降低3.56%
- 适合做3D神经元分割的科研人员和深度学习开发者
从3D光显微镜成像数据中重建神经元形态对解析脑网络与神经解剖至关重要。借助深度学习，已有多种基于学习的分割模型用于增强原始神经元图像的信噪比，作为重建流程的预处理步骤。然而，现有模型大多直接编码体数据的潜在特征，忽视其内在形态学知识。为此，我们设计了一种新框架，将大规模2D自然图像上预训练的视觉变换器知识，迁移到3D视觉变换器中以促进神经元形态学习。为弥合2D自然图像与3D微观形态域之间的知识鸿沟，提出可变形管状转移策略，在潜在嵌入空间中适应神经元结构的固有管状特性。在BigNeuron项目中的Janelia数据集上的实验表明，本方法在平均Dice上提升4.53%，平均95% Hausdorff距离降低3.56%。
原文摘要 · Abstract (English)
Reconstructing neuron morphology from 3D light microscope imaging data is critical to aid neuroscientists in analyzing brain networks and neuroanatomy. With the boost from deep learning techniques, a variety of learning-based segmentation models have been developed to enhance the signal-to-noise ratio of raw neuron images as a pre-processing step in the reconstruction workflow. However, most existing models directly encode the latent representative features of volumetric neuron data but neglect their intrinsic morphological knowledge. To address this limitation, we design a novel framework that distills the prior knowledge from a 2D Vision Transformer pre-trained on extensive 2D natural images to facilitate neuronal morphological learning of our 3D Vision Transformer. To bridge the knowledge gap between the 2D natural image and 3D microscopic morphologic domains, we propose a deformable tubular transferring strategy that adapts the pre-trained 2D natural knowledge to the inherent tubular characteristics of neuronal structure in the latent embedding space. The experimental results on the Janelia dataset of the BigNeuron project demonstrate that our method achieves a segmentation performance improvement of 4.53% in mean Dice and 3.56% in mean 95% Hausdorff distance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。