arXiv:2412.12226cs.LGcs.AI2024-12被引 2

用新方法解决时序预测中信号失真和推理慢的问题。

Apollo-Forecast: Overcoming Aliasing and Inference Speed Challenges in Language Models for Time Series Forecasting

  • 设计抗混叠量化模块,减少信号噪声,提升编码精度。
  • 引入并行解码技术,使长序列预测速度提升1.9到2.7倍。
  • 在零样本场景下显著优于现有方法,适合大规模时序建模。

将时间序列编码为令牌并使用语言模型进行处理,可显著提升模型对未见任务的泛化能力。然而,现有的时序语言模型面临混叠失真和推理时间过长的问题,主要源于量化过程限制和大模型计算开销。本文提出Apollo-Forecast框架,包含两项关键创新:抗混叠量化模块(AAQM)与竞速解码(RD)技术。AAQM在序列编码中有效抑制原始信号中的高频噪声,提升信号保真度与量化效率;RD利用草稿模型实现并行处理与结果融合,大幅加速长期预测的推理速度,尤其适用于大模型。在多个真实数据集上的实验表明,Apollo-Forecast在零样本场景下,相比最先进方法在WQL和MASE指标上分别提升35.41%和18.99%;推理速度相较基线方法提升1.9X-2.7X。

原文摘要 · Abstract (English)

Encoding time series into tokens and using language models for processing has been shown to substantially augment the models' ability to generalize to unseen tasks. However, existing language models for time series forecasting encounter several obstacles, including aliasing distortion and prolonged inference times, primarily due to the limitations of quantization processes and the computational demands of large models. This paper introduces Apollo-Forecast, a novel framework that tackles these challenges with two key innovations: the Anti-Aliasing Quantization Module (AAQM) and the Race Decoding (RD) technique. AAQM adeptly encodes sequences into tokens while mitigating high-frequency noise in the original signals, thus enhancing both signal fidelity and overall quantization efficiency. RD employs a draft model to enable parallel processing and results integration, which markedly accelerates the inference speed for long-term predictions, particularly in large-scale models. Extensive experiments on various real-world datasets show that Apollo-Forecast outperforms state-of-the-art methods by 35.41\% and 18.99\% in WQL and MASE metrics, respectively, in zero-shot scenarios. Furthermore, our method achieves a 1.9X-2.7X acceleration in inference speed over baseline methods.

时序预测语言模型量化加速推理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。