arXiv:2510.12847cs.LG2025-10被引 2

解决时序大模型伪对齐问题,提升预测性能

Lifting Manifolds to Mitigate Pseudo-Alignment in LLM4TS

  • 通过提升时序数据流形维度缓解预训练语言模型的锥形效应
  • 在多个时序预测任务上超越现有SOTA方法,长周期预测效果显著
  • 可无缝嵌入四种主流时序大模型框架,适合追求高精度预测的研究者

伪对齐是诸多大语言模型用于时间序列(LLM4TS)中的普遍挑战,常导致模型表现低于线性模型或随机初始化的主干网络。本文深入探究了伪对齐的根本原因,发现其源于预训练语言模型组件内部的锥形效应与时间序列数据固有的低维流形之间的相互作用。为此,我们提出新方法TimeSUP,通过提升时序数据流形维度以更贴近语言嵌入的内在维度,使模型能清晰区分时序信号,同时保留跨模态共享结构。结果表明,时间与语言标记的表示既保持差异性,又具备高余弦相似度,实现模态特异性与统一嵌入空间的平衡。实验显示,TimeSUP在长期预测任务中持续优于当前SOTA方法及其他轻量级基线,并可无缝集成至四种现有LLM4TS流水线,带来显著性能提升。

原文摘要 · Abstract (English)

Pseudo-Alignment is a pervasive challenge in many large language models for time series (LLM4TS) models, often causing them to underperform compared to linear models or randomly initialised backbones. However, there is limited discussion in the community for the reasons that pseudo-alignment occurs. In this work, we conduct a thorough investigation into the root causes of pseudo-alignment in LLM4TS and build a connection of pseudo-alignment to the cone effect in LLM. We demonstrate that pseudo-alignment arises from the interplay of cone effect within pretrained LLM components and the intrinsically low-dimensional manifold of time-series data. In addition, we also introduce \textit{\textbf{TimeSUP}}, a novel technique designed to mitigate this issue and improve forecast performance in existing LLM4TS approaches. TimeSUP addresses this by increasing the time series manifold to more closely match the intrinsic dimension of language embeddings, allowing the model to distinguish temporal signals clearly while still capturing shared structures across modalities. As a result, representations for time and language tokens remain distinct yet exhibit high cosine similarity, signifying that the model preserves each modality unique features while learning their commonalities in a unified embedding space. Empirically, TimeSUP consistently outperforms state-of-the-art LLM4TS methods and other lightweight baselines on long-term forecasting performance. Furthermore, it can be seamlessly integrated into four existing LLM4TS pipelines and delivers significant improvements in forecasting performance.

时序预测大模型伪对齐流形学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。