arXiv:2412.20790cs.LGcs.AI2024-12AAAI被引 13

提出无需正负样本的时序表示学习新方法,提升模型泛化能力。

Frequency-Masked Embedding Inference: A Non-Contrastive Approach for Time Series Representation Learning

  • 用频域掩码作为提示,构建双分支推理机制
  • 在8个数据集上线性评估表现超越对比学习方法
  • 适合需要高效自监督训练的时序分析场景

对比学习支撑了当前大多数自监督时序表示学习方法。正负样本对的构建策略显著影响最终表示质量。然而,由于时序语义具有连续性,对比学习建模方式难以适应时序数据特性,导致难负样本构建困难,并在正样本构造中引入不当偏差。尽管近期研究提出了若干有效策略改进正负样本构造,仍受限于对比学习框架。为根本克服对比学习局限,本文提出频率掩码嵌入推理(FEI),一种完全消除正负样本需求的非对比方法。FEI基于提示策略构建两个推理分支:1)以频域掩码为提示,推断缺失频段的目标序列嵌入表示;2)以目标序列为提示,推断其频域掩码嵌入。该方法实现时序数据的连续语义关系建模。在8个常用时序数据集上的分类与回归任务实验中,采用线性评估和端到端微调,结果表明FEI在泛化性能上显著优于现有对比学习方法。本研究为时序自监督表示学习提供了新思路。代码已开源:https://github.com/USTBInnovationPark/Frequency-masked-Embedding-Inference。

原文摘要 · Abstract (English)

Contrastive learning underpins most current self-supervised time series representation methods. The strategy for constructing positive and negative sample pairs significantly affects the final representation quality. However, due to the continuous nature of time series semantics, the modeling approach of contrastive learning struggles to accommodate the characteristics of time series data. This results in issues such as difficulties in constructing hard negative samples and the potential introduction of inappropriate biases during positive sample construction. Although some recent works have developed several scientific strategies for constructing positive and negative sample pairs with improved effectiveness, they remain constrained by the contrastive learning framework. To fundamentally overcome the limitations of contrastive learning, this paper introduces Frequency-masked Embedding Inference (FEI), a novel non-contrastive method that completely eliminates the need for positive and negative samples. The proposed FEI constructs 2 inference branches based on a prompting strategy: 1) Using frequency masking as prompts to infer the embedding representation of the target series with missing frequency bands in the embedding space, and 2) Using the target series as prompts to infer its frequency masking embedding. In this way, FEI enables continuous semantic relationship modeling for time series. Experiments on 8 widely used time series datasets for classification and regression tasks, using linear evaluation and end-to-end fine-tuning, show that FEI significantly outperforms existing contrastive-based methods in terms of generalization. This study provides new insights into self-supervised representation learning for time series. The code is available at https://github.com/USTBInnovationPark/Frequency-masked-Embedding-Inference.

时序表示学习非对比学习频域掩码自监督

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。