Mantis用合成数据训练时间序列分类模型，性能超越现有方法。
Mantis: Lightweight Foundation Model for Time Series Classification
- 基于自监督对比学习，用合成数据预训练时间序列变换器。
- 通过中间层表示与融合策略，测试时性能接近专用模型。
- 适用于多领域时间序列分类，尤其适合数据少的场景。
尽管基础模型已革新多个领域，其在时间序列分类中的应用仍不充分，现有研究多集中于预测任务。为此，我们提出基于Transformer的Mantis基础模型，仅通过自监督对比学习在合成数据上进行预训练。实验表明，有效的分词对释放Transformer潜力至关重要，因此我们设计了一种新型分词生成单元。此外，引入增强的测试时方法，利用中间层表示、自集成与跨模型嵌入融合，弥合了Mantis与强专用方法之间的性能差距。大量实验证明，Mantis在涵盖多个应用领域的四个不同数据集集合上均达到新基准，优于现有基础模型。
原文摘要 · Abstract (English)
While foundation models have revolutionized various domains, their application to time series classification remains rather under-explored, with existing literature predominantly focused on forecasting. To bridge this gap, we introduce \textbf{Mantis}, a transformer-based foundation model pre-trained exclusively on synthetic data via self-supervised contrastive learning. We demonstrate that effective tokenization is critical to unlocking the full potential of transformers, proposing a novel token generator unit. Furthermore, we introduce an enhanced test-time methodology that bridges the performance gap between Mantis and strong specialized approaches by leveraging intermediate-layer representations, self-ensembling, and cross-model embedding fusion. Extensive experiments demonstrate that Mantis establishes a new state-of-the-art, outperforming existing foundation models across four diverse dataset collections covering various application domains.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。