arXiv:2508.20622cs.LGcs.CV2025-08

用掩码自编码器学习超声信号表示,提升下游任务性能。

Masked Autoencoders for Ultrasound Signals: Robust Representation Learning for Downstream Applications

  • 用ViT架构的掩码自编码器对一维超声信号进行自监督预训练
  • 合成数据预训练后,在飞行时间分类任务上显著优于从零开始训练的模型
  • 适用于标注数据少的工业无损检测场景,尤其适合迁移至真实信号

我们研究了基于视觉变压器(ViT)架构的掩码自编码器(MAE)在一类一维超声信号上的自监督表征学习适应性与性能。尽管MAE在计算机视觉等领域已取得显著成功,但其在原始超声信号等一维信号分析中的应用仍不充分。超声信号广泛用于工业无损检测(NDT)和结构健康监测(SHM),而这类任务中标签数据稀缺且信号处理高度依赖具体任务。本文提出一种方法:利用大量未标注的合成超声信号进行MAE预训练,使模型学习鲁棒表征,从而提升下游任务如飞行时间(ToF)分类的表现。系统评估了模型规模、分块大小和遮蔽率对预训练效率与下游准确率的影响。结果表明,预训练模型显著优于从零开始训练的模型及针对下游任务优化的强卷积神经网络(CNN)基线。此外,仅在合成数据上预训练的模型在迁移到真实测量信号时,表现优于仅在有限真实数据上训练的模型。本研究证明了MAE在可扩展的自监督学习框架下,推动超声信号分析的潜力。

原文摘要 · Abstract (English)

We investigated the adaptation and performance of Masked Autoencoders (MAEs) with Vision Transformer (ViT) architectures for self-supervised representation learning on one-dimensional (1D) ultrasound signals. Although MAEs have demonstrated significant success in computer vision and other domains, their use for 1D signal analysis, especially for raw ultrasound data, remains largely unexplored. Ultrasound signals are vital in industrial applications such as non-destructive testing (NDT) and structural health monitoring (SHM), where labeled data are often scarce and signal processing is highly task-specific. We propose an approach that leverages MAE to pre-train on unlabeled synthetic ultrasound signals, enabling the model to learn robust representations that enhance performance in downstream tasks, such as time-of-flight (ToF) classification. This study systematically investigated the impact of model size, patch size, and masking ratio on pre-training efficiency and downstream accuracy. Our results show that pre-trained models significantly outperform models trained from scratch and strong convolutional neural network (CNN) baselines optimized for the downstream task. Additionally, pre-training on synthetic data demonstrates superior transferability to real-world measured signals compared with training solely on limited real datasets. This study underscores the potential of MAEs for advancing ultrasound signal analysis through scalable, self-supervised learning.

自监督学习超声信号表征学习掩码自编码器

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。