提出时序版联合嵌入模型，提升时间序列表征学习鲁棒性。
Joint Embeddings Go Temporal
- 在隐空间中进行自监督学习，避免传统方法对噪声敏感。
- 在分类与预测任务上表现媲美或超越当前最佳基线。
- 适合构建通用时间序列基础模型，适用于多种下游任务。
自监督学习在自然语言和图像处理领域取得了显著进展，但现有方法多依赖自回归或掩码建模，易受噪声和混淆变量影响。为此，本文提出针对时间序列的联合嵌入预测架构（TS-JEPA），将联合嵌入思想应用于时序数据表征学习。我们在多个标准数据集上验证了该方法在分类与预测任务上的有效性，结果表明其性能可达到或超越当前最优基线。更重要的是，该方法在不同任务间表现出良好的性能平衡，展现出作为通用时间序列表征学习基础模型的巨大潜力。本工作为未来基于联合嵌入的时间序列基础模型研发奠定了重要基础。
原文摘要 · Abstract (English)
Self-supervised learning has seen great success recently in unsupervised representation learning, enabling breakthroughs in natural language and image processing. However, these methods often rely on autoregressive and masked modeling, which aim to reproduce masked information in the input, which can be vulnerable to the presence of noise or confounding variables. To address this problem, Joint-Embedding Predictive Architectures (JEPA) has been introduced with the aim to perform self-supervised learning in the latent space. To leverage these advancements in the domain of time series, we introduce Time Series JEPA (TS-JEPA), an architecture specifically adapted for time series representation learning. We validate TS-JEPA on both classification and forecasting, showing that it can match or surpass current state-of-the-art baselines on different standard datasets. Notably, our approach demonstrates a strong performance balance across diverse tasks, indicating its potential as a robust foundation for learning general representations. Thus, this work lays the groundwork for developing future time series foundation models based on Joint Embedding.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。