让电磁信号模型在噪声中仍能准确理解,提升智能分析能力。
MERLIN: Building Low-SNR Robust Multimodal LLMs for Electromagnetic Signals
- 构建低信噪比下对齐信号与语义的训练框架
- 在10万组信号-文本数据上实现领先性能
- 适合从事电磁信号智能分析的研究者
多模态大模型为推进电磁领域带来新可能,但现有方法多采用特定任务或流水线架构,难以发挥模型潜力。核心挑战包括:(1)缺乏高质量的配对电磁信号与文本标注数据;(2)缺少系统评估模型性能的基准;(3)在低信噪比环境下模型易失效。为此,我们提出三项贡献:首先,构建并发布包含超过10万组信号-文本对的EM-100k数据集;其次,设计涵盖感知到推理的多样化任务的综合基准EM-Bench;最后,提出MERLIN训练框架,不仅对齐信号与语义表征,更显著提升低信噪比环境下的鲁棒性。实验表明,MERLIN在EM-Bench上达到当前最优水平,并在低信噪比条件下表现优异。
原文摘要 · Abstract (English)
The paradigm of Multimodal Large Language Models (MLLMs) offers a promising blueprint for advancing the electromagnetic (EM) domain. However, prevailing approaches often deviate from the native MLLM paradigm, instead using task-specific or pipelined architectures that lead to fundamental limitations in model performance and generalization. Fully realizing the MLLM potential in EM domain requires overcoming three main challenges: (1) Data. The scarcity of high-quality datasets with paired EM signals and descriptive text annotations used for MLLMs pre-training; (2) Benchmark. The absence of comprehensive benchmarks to systematically evaluate and compare the performance of models on EM signal-to-text tasks; (3) Model. A critical fragility in low Signal-to-Noise Ratio (SNR) environments, where critical signal features can be obscured, leading to significant performance degradation. To address these challenges, we introduce a tripartite contribution to establish a foundation for MLLMs in the EM domain. First, to overcome data scarcity, we construct and release EM-100k, a large-scale dataset comprising over 100,000 EM signal-text pairs. Second, to enable rigorous and standardized evaluation, we propose EM-Bench, the most comprehensive benchmark featuring diverse downstream tasks spanning from perception to reasoning. Finally, to tackle the core modeling challenge, we present MERLIN, a novel training framework designed not only to align low-level signal representations with high-level semantic text, but also to explicitly enhance model robustness and performance in challenging low-SNR environments. Comprehensive experiments validate our method, showing that MERLIN is state-of-the-art in the EM-Bench and exhibits remarkable robustness in low-SNR settings.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。