用变化检测提升电子显微镜图像分割精度,尤其擅长小而模糊的结构。
MultiTaskDeltaNet: Change Detection-based Image Segmentation for Operando ETEM with Application to Carbon Gasification Kinetics
- 将分割任务转为变化检测,用成对图像和双路网络捕捉动态变化
- 在碳气化实验中,小结构识别准确率比传统模型高10.22%
- 适合缺乏标注数据的纳米材料实时分析,助力原位实验研究
将原位透射电子显微镜(TEM)成像转化为固态反应空间分辨的原位表征工具,需实现动态演化特征的高精度语义分割。然而,传统深度学习分割方法常受限于标注数据稀缺、目标特征视觉模糊及小物体场景。为此,本文提出MultiTaskDeltaNet(MTDN),一种创新的深度学习架构,将分割任务重构为变化检测问题。通过采用具有U-Net主干的特殊孪生网络,并利用成对图像捕捉特征变化,MTDN能以极少标注数据生成高质量分割结果。此外,该模型采用多任务学习策略,挖掘不同物理特征间的关联性。在纤维状碳气化过程的原位环境TEM视频数据上评估显示,相较于传统分割模型,MTDN在精确划分细微结构方面表现显著更优,尤其在识别小而模糊的物理特征时,性能提升达10.22%。本工作弥合了深度学习与实际TEM图像分析之间的多个关键差距,推动了复杂实验环境下纳米材料自动化表征的发展。
原文摘要 · Abstract (English)
Transforming in-situ transmission electron microscopy (TEM) imaging into a tool for spatially-resolved operando characterization of solid-state reactions requires automated, high-precision semantic segmentation of dynamically evolving features. However, traditional deep learning methods for semantic segmentation often encounter limitations due to the scarcity of labeled data, visually ambiguous features of interest, and small-object scenarios. To tackle these challenges, we introduce MultiTaskDeltaNet (MTDN), a novel deep learning architecture that creatively reconceptualizes the segmentation task as a change detection problem. By implementing a unique Siamese network with a U-Net backbone and using paired images to capture feature changes, MTDN effectively utilizes minimal data to produce high-quality segmentations. Furthermore, MTDN utilizes a multi-task learning strategy to leverage correlations between physical features of interest. In an evaluation using data from in-situ environmental TEM (ETEM) videos of filamentous carbon gasification, MTDN demonstrated a significant advantage over conventional segmentation models, particularly in accurately delineating fine structural features. Notably, MTDN achieved a 10.22% performance improvement over conventional segmentation models in predicting small and visually ambiguous physical features. This work bridges several key gaps between deep learning and practical TEM image analysis, advancing automated characterization of nanomaterials in complex experimental settings.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。