针对不规则多变量时间序列分类，提出多尺度令牌混合注意力模型
MTM: A Multi-Scale Token Mixing Transformer for Irregular Multivariate Time Series Classification
- 通过降采样与掩码拼接池化，缓解通道异步问题
- 在真实数据集上提升3.8%的AUPRC，优于现有方法
- 适合医疗、金融等存在异步观测的时序分析场景
不规则多变量时间序列（IMTS）的特点是各通道观测时间不同步。本文指出，这种通道异步性会损害现有深度学习方法的通道建模能力。为此，提出MTM：一种用于IMTS分类的多尺度令牌混合变压器。发现将时间序列降采样至更粗粒度时间尺度可缓解通道异步，因此在MTM中引入掩码拼接池化，逐步降采样以增强通道注意力模块。同时，提出新型通道内令牌混合机制，主动选择某通道的重要令牌并与其他通道混合，进一步提升通道学习效果。在多个真实世界数据集上的大量实验表明，MTM在所有基准上均表现最优，分类任务中AUPRC最高提升达3.8%。
原文摘要 · Abstract (English)
Irregular multivariate time series (IMTS) is characterized by the lack of synchronized observations across its different channels. In this paper, we point out that this channel-wise asynchrony can lead to poor channel-wise modeling of existing deep learning methods. To overcome this limitation, we propose MTM, a multi-scale token mixing transformer for the classification of IMTS. We find that the channel-wise asynchrony can be alleviated by down-sampling the time series to coarser timescales, and propose to incorporate a masked concat pooling in MTM that gradually down-samples IMTS to enhance the channel-wise attention modules. Meanwhile, we propose a novel channel-wise token mixing mechanism which proactively chooses important tokens from one channel and mixes them with other channels, to further boost the channel-wise learning of our model. Through extensive experiments on real-world datasets and comparison with state-of-the-art methods, we demonstrate that MTM consistently achieves the best performance on all the benchmarks, with improvements of up to 3.8% in AUPRC for classification.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。