提出无需数据增强的时序自监督学习方法,提升模型泛化能力。
LeNEPA: No-Augmentation Next-Latent Prediction for Time-Series Representation Learning
- 采用因果主干和轻量投影空间,无须数据增强进行下个隐状态预测
- 在多个数据集上保持稳定性能,2-5千次更新即达80%效果提升
- 适合追求低调参成本的时序表征学习场景
时序数据在工业遥测、服务器指标、金融和生理信号中至关重要,但现有自监督学习依赖特定领域增强策略。本文研究当预训练信号变化时,固定方法配置的表现,将其作为固定配方压力测试。提出无增强的下隐状态预测架构LeNEPA,用SIGReg正则替代传统stop-gradient/EMA,并在轻量投影空间计算损失。在PTB-XL与Diag数据集上,使用固定时间窗冻结探针协议比较,ECG调优的JEPA在跨域表现下降,而LeNEPA在两数据集均保持有效表征。学习曲线显示,LeNEPA在2–5k更新内达80%最终性能,快于最快JEPA读出(5–10k)。另验证变体在UCR-128上达77.65%随机森林准确率,仅差于Mantis 1.16点,近于MOMENT 0.24点。结果支持无增强隐状态预测作为低调参成本的时序自监督学习候选方案。
原文摘要 · Abstract (English)
Time series are central to modern data mining applications, from industrial telemetry and server metrics to finance and physiology, yet time-series self-supervised learning often depends on view and augmentation choices that encode domain-specific invariances. We study how an SSL recipe behaves when its method-specific configuration is reused unchanged after the pretraining signal family changes, framing this as a fixed-recipe stress test rather than a comparison against optimally tuned methods. We introduce Latent Euclidean Next-Embedding Prediction Architecture (LeNEPA), a no-augmentation next-latent-token objective with a causal backbone. LeNEPA replaces the stop-gradient/EMA stabilization used by vanilla NEPA with SIGReg-based isotropy regularization and computes the predictive loss in a lightweight projected space that is discarded for evaluation. We compare LeNEPA with an ECG-tuned JEPA recipe under a fixed-horizon frozen-probe protocol on PTB-XL and Diag, a synthetic diagnostic corpus generated with Aionoscope. Both methods are retrained independently on each dataset while keeping their method-specific recipes unchanged. In this protocol, the ECG-tuned JEPA recipe is strong in-domain on PTB-XL but weaker when reused unchanged on Diag, whereas LeNEPA preserves useful frozen-probe gains on both datasets. Learning curves suggest faster early representation acquisition: LeNEPA reaches 80% of its final AUROC/AUPRC gain after 2--5k updates, compared with 5--10k updates for the faster JEPA readout. As a separate external frozen-encoder check, a CauKer-pretrained LeNEPA variant reaches 77.65% mean UCR-128 Random-Forest accuracy in a single-seed, best-checkpoint run, within 1.16 points of Mantis and within 0.24 points of MOMENT (77.89%). Overall, the results support no-augmentation latent prediction as a useful candidate recipe for low-retuning time-series SSL.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。