让大模型学会理解时间序列数据,实现文本与数值分析的融合。
TsLLM: Augmenting LLMs for General Time Series Understanding and Prediction
- 用分块编码器增强大模型的时间序列感知能力
- 在250亿个标记数据上训练,统一处理预测与自然语言任务
- 零样本和少样本表现优异,适合跨领域时序分析
时间序列数据在医疗、金融、电力系统和物流等领域对决策至关重要。但正确分析这类数据通常需要结合非结构化上下文信息、回答领域特定问题并生成自然语言解释,而传统时间序列模型缺乏这些能力。尽管大语言模型(LLM)擅长上下文推理与知识整合,却因基于文本的表示方式低效且预训练中对数值数据接触有限,难以处理时间序列。为此,我们通过分块编码器-解码器架构,将专用时间序列感知能力注入大模型,构建了时间序列增强型大模型(TsLLM)。该模型在超过250亿个标记的混合时间序列与文本数据上进行训练,涵盖预测、问答、异常检测、分类、报告生成等多样化任务,统一为下一词预测目标。训练使TsLLM兼具语言理解与新获得的时序推理能力。虽未针对传统基准优化性能,但在需融合时间序列分析与自然语言的任务上表现强劲,展现出优秀的零样本和少样本适应能力,无需额外训练即可应对新数据。
原文摘要 · Abstract (English)
Time series data is fundamental to decision-making across many domains including healthcare, finance, power systems, and logistics. However, analyzing this data correctly often requires incorporating unstructured contextual information, answering domain-specific questions, and generating natural language explanations - capabilities that traditional time series models lack. While Large Language Models (LLMs) excel at contextual reasoning and knowledge integration, they struggle with numerical time series due to inefficient text-based representations and limited exposure to numerical data during pretraining. We address this gap by augmenting an LLM with specialized time series perception through a patch-based encoder-decoder architecture. We train this Time Series augmented LLM (TsLLM) on a large corpus of over 25 billion tokens of interleaved time series and text spanning diverse tasks: forecasting with contextual information, question-answering, anomaly detection, classification, report generation, and more, all unified as next token prediction. This training enables TsLLM to leverage both its language understanding and newly acquired temporal reasoning capabilities. While not designed to surpass specialized models on traditional benchmarks, TsLLM demonstrates strong performance on tasks requiring the integration of time series analysis with natural language - capabilities that existing approaches cannot provide. It also exhibits strong zero-shot and few-shot performance, showing it can adapt to new data without additional training.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。