语言预训练让大模型具备时间序列预测能力，因提前构建了可复用的几何结构。
LLM Pretraining Shapes a Generalizable Manifold: Insights into Cross-Modal Transfer to Time Series
- 用冻结的LLM状态做线性探测，无需配对数据即可解码真实时间序列轨迹。
- 预训练初始化使优化更稳定，损失曲面高度各向异性，优于随机初始化。
- 微调仅需低秩更新，复用已有周期性、趋势等特征，适合高效迁移学习。
语言预训练的Transformer能否成为有效的时间序列预测器？本文表明，跨模态迁移源于语言预训练为时间序列训练预设了一个可复用的流形结构。在冻结的LLM状态上使用线性探测，无需配对监督即可解码真实时间序列轨迹；在该投影空间中进行检索，能获得具有竞争力的预测结果，说明在微调前已存在结构与动态。预训练初始化还改善了优化过程，产生连贯梯度和高度各向异性的损失曲面，不同于随机初始化。微调阶段相当于低维对齐，复用已有方向而非从头学习时序特征，表现为低秩更新、子空间对齐以及周期性、趋势和重复性的共享特征。这些结果支持一种几何视角：语言预训练构建流形，微调将数值动态投影到任务相关方向。
原文摘要 · Abstract (English)
Can language-pretrained transformers become effective time-series forecasters, and why? In this paper, we show that cross-modal transfer arises because language pretraining preconditions time series training with a reusable manifold. A linear probe on frozen LLM states decodes realistic time-series trajectories without paired supervision, and retrieval in this projected space yields competitive forecasts, showing that structure and dynamics exist before finetuning. Pretrained initialization also improves optimization, producing coherent gradients and a highly anisotropic loss landscape unlike random initialization. Finetuning then acts as low-dimensional alignment, reusing existing directions rather than learning temporal primitives from scratch, as evidenced by low-rank updates, subspace alignment, and shared features for periodicity, trend, and repetition. Together, these results support a geometric account of LLM-to-time-series transfer: language pretraining builds the manifold, and finetuning projects numerical dynamics onto task-relevant directions.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。