用大模型将时间序列转为符号令牌,实现带上下文的精准预测。
From Values to Tokens: An LLM-Driven Framework for Context-aware Time Series Forecasting via Symbolic Discretization
- 将数值序列转为语言式符号令牌,统一处理时序与文本
- 在共享语义空间中对齐多模态数据,提升预测准确率
- 适合需要结合文本上下文的时间序列场景,如金融、医疗
时间序列预测在能源、医疗、金融等关键领域至关重要。尽管近年取得进展,但因难以融合历史数值序列与非结构化文本类上下文特征,预测精度仍受限。为此,我们提出TokenCast框架,利用大语言模型(LLM)驱动的符号化表示作为统一中介,实现上下文感知的时间序列预测。具体而言,该框架采用离散分词器将连续数值序列转化为时间令牌,实现与语言输入的结构对齐;通过预训练大模型将时间与上下文令牌嵌入共享表示空间,并以生成目标优化对齐效果;在此基础上,微调已对齐的LLM进行未来时间令牌预测,再解码回原始数值空间。在真实世界数据集上的大量实验验证了该框架的有效性,展现出作为生成式上下文感知预测框架的潜力。代码已公开于 https://github.com/Xiaoyu-Tao/TokenCast。
原文摘要 · Abstract (English)
Time series forecasting plays a vital role in supporting decision-making across a wide range of critical applications, including energy, healthcare, and finance. Despite recent advances, forecasting accuracy remains limited due to the challenge of integrating historical numerical sequences with contextual features, which often comprise unstructured textual data. To address this challenge, we propose TokenCast, a large language model (LLM) driven framework that leverages language-based symbolic representations as a unified intermediary for context-aware time series forecasting. Specifically, TokenCast employs a discrete tokenizer to transform continuous numerical sequences into temporal tokens, enabling structural alignment with language-based inputs. To effectively bridge the semantic gap between modalities, both temporal and contextual tokens are embedded into a shared representation space via a pre-trained LLM, further optimized with generative objectives. Building upon this unified semantic space, the aligned LLM is subsequently fine-tuned in a supervised manner to predict future temporal tokens, which are then decoded back into the original numerical space. Extensive experiments on real-world datasets demonstrate the effectiveness of our framework and highlight its potential as a generative framework for context-aware time series forecasting. The code is available at https://github.com/Xiaoyu-Tao/TokenCast.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。