用分步LoRA提升时间序列预测表达能力，解决多步预测瓶颈
Mixture of Low Rank Adaptation with Partial Parameter Sharing for Time Series Forecasting
- 分两阶段训练：先预训练单步预测模型，再用步骤特异性LoRA适配
- 引入可自适应加权的多专家LoRA（MoLA），实现跨步参数共享与性能提升
- 在多个数据集上超越现有方法，适合需要高精度多步预测的场景
多任务时间序列预测已成为主流方法，但我们发现其存在表达力瓶颈——不同时间步的预测共享相同表示，即使使用最优表示也无法避免误差。为此，我们提出两阶段框架：首先预训练一个一步预测的基础模型；随后使用步骤特异性的低秩适配（LoRA）模块进行微调。该设计使基础模型可处理任意预测步数，同时避免表达力瓶颈。进一步提出混合低秩适配（MoLA）模型，通过自适应加权的LoRA专家实现步骤间的部分参数共享，利用预测步之间的依赖关系提升效率与性能。实验表明，MoLA显著增强模型表达能力，在多个基准数据集上优于当前最先进方法。代码已公开于 https://anonymous.4open.science/r/MoLA-BC92。
原文摘要 · Abstract (English)
Multi-task forecasting has become the standard approach for time-series forecasting (TSF). However, we show that it suffers from an Expressiveness Bottleneck, where predictions at different time steps share the same representation, leading to unavoidable errors even with optimal representations. To address this issue, we propose a two-stage framework: first, pre-train a foundation model for one-step-ahead prediction; then, adapt it using step-specific LoRA modules.This design enables the foundation model to handle any number of forecast steps while avoiding the expressiveness bottleneck. We further introduce the Mixture-of-LoRA (MoLA) model, which employs adaptively weighted LoRA experts to achieve partial parameter sharing across steps. This approach enhances both efficiency and forecasting performance by exploiting interdependencies between forecast steps. Experiments show that MoLA significantly improves model expressiveness and outperforms state-of-the-art time-series forecasting methods. Code is available at https://anonymous.4open.science/r/MoLA-BC92.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。