轻量级嵌入框架提升物联网多变量时序预测精度。
LightSAE: Parameter-Efficient and Heterogeneity-Aware Embedding for IoT Multivariate Time Series Forecasting
- 拆分共享与通道特异性嵌入，保留数据独特性。
- 低秩分解+共享门控池，参数仅增4.0%但误差降22.8%。
- 适合资源受限的物联网场景，兼容多种模型结构。
现代物联网系统产生海量异构的多变量时间序列数据。准确预测此类数据对众多应用至关重要。然而，现有方法普遍使用统一嵌入层处理所有通道，造成表征瓶颈，掩盖了有价值的通道特异性信息。为此，我们提出共享-辅助嵌入（SAE）框架，将嵌入分解为捕捉共性模式的共享基础部分和建模通道特异性偏差的辅助部分。实证发现，辅助部分呈现低秩与聚类特性，而独立嵌入中该特征不明显。据此设计轻量级嵌入模块LightSAE，通过低秩分解与共享门控组件池实现高效参数化。在9个物联网相关数据集及4种骨干网络上实验表明，LightSAE在仅增加4.0%参数量的前提下，最高实现22.8%的均方误差降低。
原文摘要 · Abstract (English)
Modern Internet of Things (IoT) systems generate massive, heterogeneous multivariate time series data. Accurate Multivariate Time Series Forecasting (MTSF) of such data is critical for numerous applications. However, existing methods almost universally employ a shared embedding layer that processes all channels identically, creating a representational bottleneck that obscures valuable channel-specific information. To address this challenge, we introduce a Shared-Auxiliary Embedding (SAE) framework that decomposes the embedding into a shared base component capturing common patterns and channel-specific auxiliary components modeling unique deviations. Within this decomposition, we \rev{empirically observe} that the auxiliary components tend to exhibit low-rank and clustering characteristics, a structural pattern that is significantly less apparent when using purely independent embeddings. Consequently, we design LightSAE, a parameter-efficient embedding module that operationalizes these observed characteristics through low-rank factorization and a shared, gated component pool. Extensive experiments across 9 IoT-related datasets and 4 backbone architectures demonstrate LightSAE's effectiveness, achieving MSE improvements of up to 22.8\% with only 4.0\% parameter increase.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。