arXiv:2606.07031cs.LG2026-06

不用掩码的时序自监督学习,靠未来预测建模时间连续性。

CF-JEPA: Mask-free forward prediction with asymmetric encoder utilization for time-series representation learning

  • 用多时序前景预测替代掩码,直接利用时间顺序作为学习信号。
  • 在126个分类数据集上平均准确率领先,预测误差降低27%。
  • 在线编码器用于分类,目标编码器用于预测,无需额外训练成本。

时序表征学习中的自监督方法主要依赖对比学习(面临正负样本构建难题)或掩码方法(破坏时间连续性)。联合嵌入预测架构(JEPA)通过在表征空间中预测而非重建原始输入提供新路径。然而现有时序JEPA仍依赖掩码,继承其连续性问题。本文提出无掩码的多时序前向预测框架(CF-JEPA):随机裁剪作为上下文视图,预测短、中、长时序未来的表征,直接利用时间序列的固有顺序作为学习信号。研究发现在线编码器与指数移动平均(EMA)目标编码器在单次训练中呈现显著不对称:在线编码器生成高秩判别特征,目标编码器生成更平滑的低秩时间特征。据此,分类任务由在线编码器完成,预测或异常检测由目标编码器承担,实现27%的多变量预测均方误差(MSE)降低,且无需额外训练成本。在126个UCR和26个UEA分类数据集、8个电力变压器温度预测基准及关键性能指标/Yahoo异常检测任务中,CF-JEPA在自监督基线中于UCR/UEA数据集上取得最高平均准确率与排名,于单变量预测中排名第二,在最近邻得分异常检测中位列第二。

原文摘要 · Abstract (English)

Self-supervised learning (SSL) for time-series representation learning is dominated by two paradigms: contrastive methods, which face challenges in constructing positive or negative pairs, and masking-based methods, which disrupt the temporal continuity of time-series signals. Joint-Embedding Predictive Architecture (JEPA) offers a promising alternative by predicting in representation space rather than reconstructing raw inputs. However, existing time-series JEPA variants still rely on masking and therefore inherit its continuity problem. Crop-based Forward JEPA (CF-JEPA) is proposed as an innovative mask-free framework that replaces masking with multi-horizon forward prediction: random crops serve as context views, and short-, mid-, and long-horizon future representations are predicted in the forward temporal direction, directly leveraging the inherent temporal ordering of time-series data as a learning signal. A strong asymmetry is also identified between the online encoder and the exponential moving average (EMA) target encoder, both produced from a single training run: the online encoder develops higher-rank discriminative features, while the EMA target encoder develops smoother, lower-rank temporal features. Exploiting this asymmetry, classification is routed to the online encoder and forecasting or anomaly detection to the EMA target encoder, achieving a 27% reduction in multivariate forecasting mean squared error (MSE) at no additional training cost. Across 126 University of California, Riverside (UCR) and 26 University of East Anglia (UEA) classification datasets, eight electricity transformer temperature forecasting benchmarks, and Key Performance Indicator /Yahoo anomaly detection, CF-JEPA achieves the highest average accuracy and rank on UCR and UEA among self-supervised baselines and ranks second on univariate forecasting and k-nearest neighbors-scored anomaly detection.

自监督学习时序预测表征学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。