arXiv:2409.14978cs.AI2024-09被引 2

无需文本数据,用分层对齐提升时序预测精度

TS-HTFA: Advancing Time Series Forecasting via Hierarchical Text-Free Alignment with Large Language Models

  • 用虚拟文本替代真实文本,避免依赖标注数据
  • 在输入、特征、输出三层次实现跨模态对齐
  • 在多个基准上达到当前最优,通用性强

尽管大语言模型在序列建模中展现出巨大潜力,但现有方法仍面临两大挑战:一是依赖大量配对文本数据,限制了应用范围;二是文本与时间序列之间存在显著模态差距,导致对齐不足、性能受限。本文提出层级无文本对齐(TS-HTFA)方法,通过自适应虚拟文本(基于QR分解词嵌入与可学习提示)取代真实文本,并在输入、特征、输出三个层面建立全面的跨模态对齐,充分挖掘大语言模型的表示能力。在多个时序预测基准上的大量实验表明,该方法显著提升了预测准确率和泛化能力,达到当前最优水平。

原文摘要 · Abstract (English)

Given the significant potential of large language models (LLMs) in sequence modeling, emerging studies have begun applying them to time-series forecasting. Despite notable progress, existing methods still face two critical challenges: 1) their reliance on large amounts of paired text data, limiting the model applicability, and 2) a substantial modality gap between text and time series, leading to insufficient alignment and suboptimal performance. In this paper, we introduce \textbf{H}ierarchical \textbf{T}ext-\textbf{F}ree \textbf{A}lignment (\textbf{TS-HTFA}), a novel method that leverages hierarchical alignment to fully exploit the representation capacity of LLMs while eliminating the dependence on text data. Specifically, we replace paired text data with adaptive virtual text based on QR decomposition word embeddings and learnable prompt. Furthermore, we establish comprehensive cross-modal alignment at three levels: input, feature, and output. Extensive experiments on multiple time-series benchmarks demonstrate that HTFA achieves state-of-the-art performance, significantly improving prediction accuracy and generalization.

时序预测大模型无文本对齐跨模态

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。