提出自监督谱图建模，用海量无线信号预训练基础模型。
Self-Supervised Radio Pre-training: Toward Foundational Models for Spectrogram Learning
- 用卷积LSTM处理时频数据，通过掩码预测实现自监督预训练。
- 在频谱预测与分割任务上表现优异，精度接近有监督方法。
- 适合做无线信号分析的通用模型，降低下游任务成本。
基础深度学习模型通常在大规模、多样化且无标签的数据集上通过自监督学习进行预训练，已在自然语言处理等领域取得显著进展。这类模型可微调用于下游任务，大幅缩短开发周期并降低训练成本，同时提升性能。本文提出一种新的自监督学习方法——掩码谱图建模（Masked Spectrogram Modeling），用于在无线信号上预训练基础深度学习模型。采用卷积LSTM架构以高效处理时频数据，基于从空中采集的无标签无线电数据集进行预训练。随后，将预训练模型微调至两个下游任务：频谱预测与分割。实验结果表明，该方法在两项任务中均表现出色，验证了其在构建无线信号基础模型方面的有效性。
原文摘要 · Abstract (English)
Foundational deep learning (DL) models are general models, trained on large, diverse, and unlabelled datasets, typically using self-supervised learning techniques have led to significant advancements especially in natural language processing. These pretrained models can be fine-tuned for related downstream tasks, offering faster development and reduced training costs, while often achieving improved performance. In this work, we introduce Masked Spectrogram Modeling, a novel self-supervised learning approach for pretraining foundational DL models on radio signals. Adopting a Convolutional LSTM architecture for efficient spatio-temporal processing, we pretrain the model with an unlabelled radio dataset collected from over-the-air measurements. Subsequently, the pretrained model is fine-tuned for two downstream tasks: spectrum forecasting and segmentation. Experimental results demonstrate that our methodology achieves competitive performance in both forecasting accuracy and segmentation, validating its effectiveness for developing foundational radio models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。