用频域融合文本与时间序列,提升多模态预测精度
Spectral Text Fusion: A Frequency-Aware Approach to Multimodal Time-Series Forecasting
- 将文本嵌入映射到频域,与时间序列的频率分量融合
- 在多个数据集上超越现有模型,参数量更少
- 适合需要结合文本上下文的时间序列预测任务
多模态时间序列预测在实际应用中至关重要,决策依赖于数值数据与上下文信号。核心挑战在于有效融合时间序列的数值模式与文本等其他模态中的上下文信息。现有方法通常逐步对齐文本特征与时间序列模式,忽略了上下文信息的多尺度时间影响,如周期性变化和动态漂移。这种局部对齐与全局文本语境之间的不匹配可通过谱分解解决,该方法将时间序列分解为捕捉短期波动与长期趋势的频率分量。本文提出SpecTF框架,通过频域融合文本对时间序列的影响:提取文本嵌入,将其投影至频域,并使用轻量级交叉注意力机制与时间序列的谱成分融合,根据文本相关性自适应重加权频率带,再映射回时域进行预测。实验表明,SpecTF在多个多模态时间序列数据集上显著优于当前最优模型,且参数量更少。代码已开源。
原文摘要 · Abstract (English)
Multimodal time series forecasting is crucial in real-world applications, where decisions depend on both numerical data and contextual signals. The core challenge is to effectively combine temporal numerical patterns with the context embedded in other modalities, such as text. While most existing methods align textual features with time-series patterns one step at a time, they neglect the multiscale temporal influences of contextual information such as time-series cycles and dynamic shifts. This mismatch between local alignment and global textual context can be addressed by spectral decomposition, which separates time series into frequency components capturing both short-term changes and long-term trends. In this paper, we propose SpecTF, a simple yet effective framework that integrates the effect of textual data on time series in the frequency domain. Our method extracts textual embeddings, projects them into the frequency domain, and fuses them with the time series' spectral components using a lightweight cross-attention mechanism. This adaptively reweights frequency bands based on textual relevance before mapping the results back to the temporal domain for predictions. Experimental results demonstrate that SpecTF significantly outperforms state-of-the-art models across diverse multi-modal time series datasets while utilizing considerably fewer parameters. Code is available at https://github.com/hiepnh137/SpecTF.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。