超轻量时序模型通过解耦表示实现零样本快速分析
TSPulse: Tiny Pre-Trained Models with Disentangled Representations for Rapid Time-Series Analysis
- 将时序信号解耦为时间、频域和语义三视图嵌入
- 在4个诊断任务上提升20%-50%,优于10-100倍大的模型
- 支持无GPU部署,适合边缘设备实时应用
时序任务常需跨表示空间(如时间与频率)和多抽象层级(如局部模式与全局语义)的信号。现有预训练时序模型将这些异构信号混杂于单一大嵌入中,限制迁移能力与零样本可用性。为此,我们提出TSPulse,一组超轻量预训练模型(仅100万参数),具备解耦特性,专用于多种时序诊断任务。TSPulse引入新颖预训练框架,通过掩码重建结合显式解耦,学习时间、频域和语义三个互补嵌入视图,有效支持零样本迁移。此外,设计轻量级后处理融合器,按任务类型选择性关注并融合这些解耦视图,实现简单有效的任务定制。为增强鲁棒性并缓解现有方法中的掩码偏差,提出一种简单但有效的混合掩码策略,提升预训练期间缺失数据多样性。尽管模型极小,TSPulse在四个时序诊断任务上均表现强劲且一致:在TSB-AD异常检测排行榜上提升20%,相似性搜索提升25%,数据填补提升50%,多变量分类提升5%-16%,在超过75个数据集上超越10-100倍大的模型。TSPulse实现顶尖零样本性能,支持高效微调,并可实现无GPU部署。模型与源码已公开于https://huggingface.co/ibm-granite/granite-timeseries-tspulse-r1。
原文摘要 · Abstract (English)
Time-series tasks often benefit from signals expressed across multiple representation spaces (e.g., time vs. frequency) and at varying abstraction levels (e.g., local patterns vs. global semantics). However, existing pre-trained time-series models entangle these heterogeneous signals into a single large embedding, limiting transferability and direct zero-shot usability. To address this, we propose TSPulse, family of ultra-light pre-trained models (1M parameters) with disentanglement properties, specialized for various time-series diagnostic tasks. TSPulse introduces a novel pre-training framework that augments masked reconstruction with explicit disentanglement across spaces and abstractions, learning three complementary embedding views (temporal, spectral, and semantic) to effectively enable zero-shot transfer. In-addition, we introduce various lightweight post-hoc fusers that selectively attend and fuse these disentangled views based on task type, enabling simple but effective task specializations. To further improve robustness and mitigate mask-induced bias prevalent in existing approaches, we propose a simple yet effective hybrid masking strategy that enhances missing diversity during pre-training. Despite its compact size, TSPulse achieves strong and consistent gains across four TS diagnostic tasks: +20% on the TSB-AD anomaly detection leaderboard, +25% on similarity search, +50% on imputation, and +5-16% on multivariate classification, outperforming models that are 10-100X larger on over 75 datasets. TSPulse delivers state-of-the-art zero-shot performance, efficient fine-tuning, and supports GPU-free deployment. Models and source code are publicly available at https://huggingface.co/ibm-granite/granite-timeseries-tspulse-r1.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。