针对医疗时间序列分类难题,提出可迁移的稀疏注意力模型。
MedSpaformer: a Transferable Transformer with Multi-granularity Token Sparsification for Medical Time Series Classification
- 用多粒度稀疏令牌机制动态聚焦关键信息
- 在7个数据集上超越13个基线,支持少样本与零样本诊断
- 适合跨数据集医疗诊断场景,尤其标签稀缺时
精准的医疗时间序列(MedTS)分类对临床诊断至关重要,但受复杂多通道时序依赖、信息冗余和标签稀缺等挑战制约。尽管基于Transformer的模型在时序分析中表现良好,但多数专为预测任务设计,未能充分捕捉MedTS特性。本文提出MedSpaformer,一种面向MedTS分类的Transformer框架。其采用基于稀疏令牌的双注意力机制,实现全局上下文建模与动态特征精炼,通过聚焦关键令牌减少冗余。该机制融合多粒度跨通道编码方案,捕获通道内与通道间的时序依赖及相关性,逐步优化任务相关信号模式。稀疏化设计使模型能灵活处理不同长度与通道数输入。此外引入自适应标签编码器,提取标签语义并缓解跨数据集标签空间错位问题,提升模型在异构医疗数据间的可迁移性。在7个医学数据集上的监督学习实验中,优于13个基线;在少样本学习中表现优异,并在同域与跨域诊断中均实现零样本能力。结果表明MedSpaformer具备强鲁棒性,可作为多样化场景下统一的MedTS分类解决方案。
原文摘要 · Abstract (English)
Accurate medical time series (MedTS) classification is essential for effective clinical diagnosis, yet remains challenging due to complex multi-channel temporal dependencies, information redundancy, and label scarcity. While transformer-based models have shown promise in time series analysis, most are designed for forecasting tasks and fail to fully exploit the unique characteristics of MedTS. In this paper, we introduce MedSpaformer, a transformer-based framework tailored for MedTS classification. It incorporates a sparse token-based dual-attention mechanism that enables global context modeling and token sparsification, allowing dynamic feature refinement by focusing on informative tokens while reducing redundancy. This mechanism is integrated into a multi-granularity cross-channel encoding scheme to capture intra- and inter-granularity temporal dependencies and inter-channel correlations, enabling progressive refinement of task-relevant patterns in medical signals. The sparsification design allows our model to flexibly accommodate inputs with variable lengths and channel dimensions. We also introduce an adaptive label encoder to extract label semantics and address cross-dataset label space misalignment. Together, these components enhance the model's transferability across heterogeneous medical datasets, which helps alleviate the challenge of label scarcity. Our model outperforms 13 baselines across 7 medical datasets under supervised learning. It also excels in few-shot learning and demonstrates zero-shot capability in both in-domain and cross-domain diagnostics. These results highlight MedSpaformer's robustness and its potential as a unified solution for MedTS classification across diverse settings.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。