对比生成与隐空间自监督学习，发现时序预训练对异常检测提升超3倍，但对预测作用有限。
Quantifying the Pre-training Dividend: Generative versus Latent Self-Supervised Learning for Time Series Foundation Models
- 用小波变换增强数据，对比生成式与隐空间对齐模型在时序上的表现
- 异常检测和分类任务中预训练收益最高达375%，预测任务收益微弱
- 模型性能不依赖数据来源，中等深度即可饱和，适合大规模合成数据训练
自监督学习在视觉和自然语言处理中的成功推动了其在时间序列领域的快速应用。然而现有研究主要聚焦于生成式范式和预测任务，尚未量化预训练表示的通用价值。本文建立可控评估框架，量化‘预训练红利’：即自监督学习在多种时序任务中的增益。系统比较生成式方法与隐空间对齐架构，提出适用于时序的LeJEPA与DINO改进版本，利用离散小波变换（DWT）增强以实现对局部波动的不变性。分析表明，预训练红利高度不对称：异常检测与分类任务中性能提升最高达375%，而预测任务中收益微弱。结果揭示表示能力非普适，受精度-不变性权衡制约，任务所需信号分辨率必须与学习目标匹配。最后，发现表示质量与数据来源无关，在中等模型深度下即趋于饱和，提示可通过大规模合成数据实现有效扩展。代码已开源：https://github.com/noammajor/Models
原文摘要 · Abstract (English)
The success of self-supervised learning (SSL) in vision and NLP has motivated its rapid adoption for time series. However, research has focused primarily on Generative paradigms and forecasting tasks, leaving the broader utility of learned representations unquantified. We establish a controlled framework to evaluate the "pre-training dividend": the value added by SSL across diverse temporal tasks. We systematically compare Generative paradigms against Latent Alignment architectures, introducing adaptations of LeJEPA and DINO for time series. These adaptations utilize Discrete Wavelet Transform (DWT) augmentations to enforce invariance to local fluctuations. Our analysis reveals that the pre-training dividend is highly asymmetric: SSL yields gains of up to 375% for anomaly detection and classification, yet remains marginal for forecasting. We demonstrate that representational utility is non-universal, governed by a precision-invariance trade-off where the specific signal resolution required by the task must align with the objective. Finally, we show that representation quality is largely independent of data origin and saturates at moderate architectural depths, suggesting a path to scaling via massive synthetic generation. Our code is available at: https://github.com/noammajor/Models
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。