简化令牌化策略更有效，能用更少参数实现强预测性能。
Rethinking Tokenization for Clinical Time Series: When Less is More
- 用控制实验对比多种临床时间序列令牌化方法。
- 时间编码无显著优势，代码序列已含足够预测信息。
- 冻结预训练编码器比可训练版本更高效，适合实际部署。
令牌化策略影响模型对电子健康记录的处理方式，但其有效性比较仍不充分。本文基于Transformer架构，在MIMIC-IV数据集上系统评估了四种临床预测任务中的令牌化方法，发现显式时间编码对下游任务无一致显著提升；值特征的重要性因任务而异，影响死亡率预测但不影响再入院预测，表明仅用代码序列即可提供足够预测信号。进一步发现，冻结的预训练代码编码器显著优于可训练版本，且参数量大幅减少。更大的临床编码器在各类任务中均表现更优，得益于冻结嵌入带来的计算开销降低。该可控评估为令牌化方法提供了更公平的比较基础，表明在多数情况下，更简单、参数高效的策略即可取得优异性能，但最优方案仍依赖具体任务。
原文摘要 · Abstract (English)
Tokenization strategies shape how models process electronic health records, yet fair comparisons of their effectiveness remain limited. We present a systematic evaluation of tokenization approaches for clinical time series modeling using transformer-based architectures, revealing task-dependent and sometimes counterintuitive findings about temporal and value feature importance. Through controlled ablations across four clinical prediction tasks on MIMIC-IV, we demonstrate that explicit time encodings provide no consistent statistically significant benefit for the evaluated downstream tasks. Value features show task-dependent importance, affecting mortality prediction but not readmission, suggesting code sequences alone can carry sufficient predictive signal. We further show that frozen pretrained code encoders dramatically outperform their trainable counterparts while requiring dramatically fewer parameters. Larger clinical encoders provide consistent improvements across tasks, benefiting from frozen embeddings that eliminate computational overhead. Our controlled evaluation enables fairer tokenization comparisons and demonstrates that simpler, parameter-efficient approaches can, in many cases, achieve strong performance, though the optimal tokenization strategy remains task-dependent.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。