arXiv:2601.12785cs.LGcs.AI2026-01中稿 · ICASSP-2026被引 8

提出首个专为时间序列大模型设计的压缩框架,显著减小模型体积且保持预测精度。

Distilling Time Series Foundation Models for Efficient Forecasting

  • 设计分时加权目标函数,平衡长短周期预测的学习强度。
  • 在多个数据集上实现参数减少150倍、推理速度提升6000倍。
  • 适合需要高效部署时间序列预测模型的研究与工业场景。

时间序列基础模型(TSFMs)通过大规模预训练实现强大预测性能,但其庞大的参数量导致部署成本高昂。尽管知识蒸馏是模型压缩的自然有效方法,但通用机器学习技术不适用于时间序列预测,因其独特特性。为此,我们提出首个专为TSFMs设计的蒸馏框架DistilTS。该框架解决两大挑战:(1) 任务难度差异,即统一权重使优化被短期预测主导,长期预测监督不足;(2) 架构差异,通过设计时间序列预测中的对齐机制缓解。DistilTS引入分时加权目标函数以平衡各预测时长的学习,采用时间对齐策略降低架构不匹配问题,从而生成紧凑模型。在多个基准测试中,DistilTS实现与完整版TSFMs相当的预测性能,同时参数减少达1/150,推理加速高达6000倍。代码已公开于:https://github.com/itsnotacie/DistilTS-ICASSP2026。

原文摘要 · Abstract (English)

Time Series foundation models (TSFMs) deliver strong forecasting performance through large-scale pretraining, but their large parameter sizes make deployment costly. While knowledge distillation offers a natural and effective approach for model compression, techniques developed for general machine learning tasks are not directly applicable to time series forecasting due to the unique characteristics. To address this, we present DistilTS, the first distillation framework specifically designed for TSFMs. DistilTS addresses two key challenges: (1) task difficulty discrepancy, specific to forecasting, where uniform weighting makes optimization dominated by easier short-term horizons, while long-term horizons receive weaker supervision; and (2) architecture discrepancy, a general challenge in distillation, for which we design an alignment mechanism in the time series forecasting. To overcome these issues, DistilTS introduces horizon-weighted objectives to balance learning across horizons, and a temporal alignment strategy that reduces architectural mismatch, enabling compact models. Experiments on multiple benchmarks demonstrate that DistilTS achieves forecasting performance comparable to full-sized TSFMs, while reducing parameters by up to 1/150 and accelerating inference by up to 6000x. Code is available at: https://github.com/itsnotacie/DistilTS-ICASSP2026.

时间序列模型压缩知识蒸馏

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。