用频域对齐和信息瓶颈提升时序数据蒸馏效果
DDTime: Dataset Distillation with Spectral Alignment and Information Bottleneck for Time-Series Forecasting
- 通过频域对齐缓解自相关带来的时序偏差
- 合成样本多样性提升,准确率相对提高30%
- 轻量级框架,适合各类时序预测模型
时序预测广泛应用,但训练需大量数据和计算资源。数据蒸馏通过生成紧凑数据集来保留原始学习能力,但在时序场景面临两大挑战:一是强自相关导致的时序偏差,使教师与学生模型间值项对齐失真;二是缺乏类别先验,合成样本多样性不足。本文提出DDTime框架,基于一阶凝聚分解,通过时序统计重构值项对齐,并引入频域对齐机制,保障谱一致性与时序保真度;同时设计受信息瓶颈启发的跨样本正则化,增强轨迹多样性并最大化信息密度。该联合目标理论兼容多种凝聚范式,支持稳定的一阶优化。在20个基准数据集及多种预测架构上实验表明,DDTime显著优于现有方法,准确率相对提升约30%,仅增加约2.49%计算开销。代码与蒸馏数据集将全部开源。
原文摘要 · Abstract (English)
Time-series forecasting is fundamental across many domains, yet training accurate models often requires large-scale datasets and substantial computational resources. Dataset distillation offers a promising alternative by synthesizing compact datasets that preserve the learning behavior of full data. However, extending dataset distillation to time-series forecasting is non-trivial due to two fundamental challenges: 1.temporal bias from strong autocorrelation, which leads to distorted value-term alignment between teacher and student models; and 2.insufficient diversity among synthetic samples, arising from the absence of explicit categorical priors to regularize trajectory variety. In this work, we propose DDTime, a lightweight and plug-in distillation framework built upon first-order condensation decomposition. To tackle Challenge 1, it revisits value-term alignment through temporal statistics and introduces a frequency-domain alignment mechanism to mitigate autocorrelation-induced bias, ensuring spectral consistency and temporal fidelity. To address Challenge 2, we further design an inter-sample regularization inspired by the information bottleneck principle, which enhances diversity and maximizes information density across synthetic trajectories. The combined objective is theoretically compatible with a wide range of condensation paradigms and supports stable first-order optimization. Extensive experiments on 20 benchmark datasets and diverse forecasting architectures demonstrate that DDTime consistently outperforms existing distillation methods, achieving about 30% relative accuracy gains while introducing about 2.49% computational overhead. All code and distilled datasets will be released.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。