用多小波嵌入让大模型更准预测时间序列数据
Speaking Numbers to LLMs: Multi-Wavelet Number Embeddings for Time Series Forecasting

- 将数字拆解为多尺度小波系数,生成逐位嵌入
- 在5个基准上超越传统方法,实现新最好性能
- 适合需要精准数值推理的时序预测场景
大型语言模型(LLMs)在融合异构文本信号方面具有优势,但其离散化的分词和嵌入接口与连续数值不匹配,常损害数值顺序性与预测可靠性。我们提出TempoWave,一种即插即用的时间小波数字接口,将每个标量观测值映射为由多小波、多尺度系数构建的逐位嵌入。通过直接覆盖标准标记表示,TempoWave以Transformer兼容形式同时展现精细局部波动与宏观全局结构,确保数值格式精度、数字身份区分度及对常见归一化操作的鲁棒性贯穿整个LLM流程。在五个富含上下文的预测基准上实验表明,TempoWave持续优于标准数值分词与其它嵌入接口,达到新最优水平。结果凸显数值接口是关键瓶颈,证明有原则的多分辨率嵌入能更好连接LLM的上下文推理与精确预测。代码已开源于https://github.com/DC-research/TempoWAVE,模型可在https://huggingface.co/Melady/TempoWAVE获取。
原文摘要 · Abstract (English)
Large language models (LLMs) are attractive for context-aware time series forecasting because they can integrate heterogeneous textual signals, yet their discrete, language-oriented tokenization and embedding interfaces are misaligned with continuous numerical values, often harming numerical ordering and forecasting reliability. We propose TempoWave, a plug-and-play temporal wavelet digit interface that maps each scalar observation into digit-wise embeddings constructed from multi-wavelet, multi-scale coefficients. By directly overriding standard token representations, TempoWave seamlessly exposes both fine-grained local fluctuations and macro global structures in a transformer-compatible form, ensuring that precise numerical formatting, distinct digit identity, and robustness to common normalization operations are maintained throughout the LLM pipeline. Experiments across five context-enriched forecasting benchmarks demonstrate that TempoWave consistently improves LLM-based forecasters over standard numeric tokenization and alternative embedding interfaces, achieving a new state-of-the-art. These results highlight the numeric interface as a key bottleneck and suggest that principled multi-resolution embeddings can better couple LLMs' contextual reasoning with precise forecasting. Our code is available at https://github.com/DC-research/TempoWAVE and our model can be accessed at https://huggingface.co/Melady/TempoWAVE.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。