解决无线信道信息预训练中的尺度与场景异构问题,提升模型泛化能力。
HeterCSI: Channel-Adaptive Heterogeneous CSI Pretraining Framework for Generalized Wireless Foundation Models
- 通过自适应分批和双掩码机制,有效处理不同尺度和场景的信道数据。
- 在12个数据集上实现无需微调的通用建模,性能优于现有方法7.19~5.27 dB。
- 训练延迟降低53%,适合6G多场景通用模型研发人员使用。
无线基础模型有望为6G网络中的信道状态信息(CSI)处理带来变革性能力,但受限于CSI在尺度和场景两个维度上的固有异构性。当前预训练方法或固定输入尺寸,或按尺度隔离训练,制约了无线基础模型的泛化与可扩展性。本文提出HeterCSI框架,通过重新理解异构CSI预训练中的梯度动态,实现训练效率与跨场景泛化能力的平衡。核心发现:尺度异构主要导致破坏性梯度干扰,而场景多样性在合理管理下可促进建设性梯度对齐。我们将异构CSI批量构建建模为最小化零填充开销同时保持场景多样性的分区优化问题,设计了尺度感知的自适应分批策略,并引入双掩码机制以分离有效信号与填充伪影。在12个数据集上的大量实验表明,HeterCSI无需场景特定微调即可建立通用基础模型,平均性能超越全量样本基线。相比最先进的零样本基准WiFo,其在信道重建、时域与频域预测任务中分别降低NMSE 7.19 dB、4.08 dB、5.27 dB。该框架相较现有方法减少53%训练延迟,平均泛化性能提升1.53 dB。
原文摘要 · Abstract (English)
Wireless foundation models promise transformative capabilities for channel state information (CSI) processing across diverse 6G network applications, yet face fundamental challenges due to the inherent dual heterogeneity of CSI across both scale and scenario dimensions. However, current pretraining approaches either constrain inputs to fixed dimensions or isolate training by scale, limiting the generalization and scalability of wireless foundation models. In this paper, we propose HeterCSI, a channel-adaptive pretraining framework that reconciles training efficiency with robust cross-scenario generalization via a new understanding of gradient dynamics in heterogeneous CSI pretraining. Our key insight reveals that CSI scale heterogeneity primarily causes destructive gradient interference, while scenario diversity actually promotes constructive gradient alignment when properly managed. Specifically, we formulate heterogeneous CSI batch construction as a partitioning optimization problem that minimizes zero-padding overhead while preserving scenario diversity. To solve this, we develop a scale-aware adaptive batching strategy that aligns CSI samples of similar scales, and design a double-masking mechanism to isolate valid signals from padding artifacts. Extensive experiments on 12 datasets demonstrate that HeterCSI establishes a generalized foundation model without scenario-specific finetuning, achieving superior average performance over full-shot baselines. Compared to the state-of-the-art zero-shot benchmark WiFo, it reduces NMSE by 7.19 dB, 4.08 dB, and 5.27 dB for CSI reconstruction, time-domain, and frequency-domain prediction, respectively. The proposed HeterCSI framework also reduces training latency by 53% compared to existing approaches while improving generalization performance by 1.53 dB on average.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。