arXiv:2601.16516cs.LG2026-01中稿 · ICASSP 2026被引 1

改进编码器设计可显著提升大模型对不规则重症监护数据的分类效果。

Rethinking Large Language Models For Irregular Time Series Classification In Critical Care

  • 设计显式建模数据缺失特性的编码器,提升模型对不规则时间序列的适应能力。
  • 新编码器使AUPRC平均提升12.8%,优于原始Transformer架构。
  • 适合关注医疗时间序列建模的开发者,尤其关注如何处理缺失数据。

重症监护室(ICU)的时间序列数据对患者监测至关重要。尽管大语言模型(LLM)在时间序列建模(TSM)中展现巨大潜力,但其在高缺失率的不规则ICU数据上的表现仍缺乏研究。本文系统评估了两个关键组件:时间序列编码器与多模态对齐策略。在多个基准ICU数据集上对比多种先进LLM方法与强监督及自监督基线,结果表明编码器设计比对齐策略更为重要。显式建模不规则性的编码器带来显著性能提升,平均AUPRC提高12.8%;而最优语义融合对齐策略仅比交叉注意力提升2.9%。然而,基于LLM的方法训练时间至少是最佳不规则监督模型的10倍,性能却仅相当,且在少样本学习场景下表现更差。这些发现揭示了LLM在不规则ICU时间序列中的潜力与局限。代码已公开于https://github.com/mHealthUnimelb/LLMTS。

原文摘要 · Abstract (English)

Time series data from the Intensive Care Unit (ICU) provides critical information for patient monitoring. While recent advancements in applying Large Language Models (LLMs) to time series modeling (TSM) have shown great promise, their effectiveness on the irregular ICU data, characterized by particularly high rates of missing values, remains largely unexplored. This work investigates two key components underlying the success of LLMs for TSM: the time series encoder and the multimodal alignment strategy. To this end, we establish a systematic testbed to evaluate their impact across various state-of-the-art LLM-based methods on benchmark ICU datasets against strong supervised and self-supervised baselines. Results reveal that the encoder design is more critical than the alignment strategy. Encoders that explicitly model irregularity achieve substantial performance gains, yielding an average AUPRC increase of $12.8\%$ over the vanilla Transformer. While less impactful, the alignment strategy is also noteworthy, with the best-performing semantically rich, fusion-based strategy achieving a modest $2.9\%$ improvement over cross-attention. However, LLM-based methods require at least 10$\times$ longer training than the best-performing irregular supervised models, while delivering only comparable performance. They also underperform in data-scarce few-shot learning settings. These findings highlight both the promise and current limitations of LLMs for irregular ICU time series. The code is available at https://github.com/mHealthUnimelb/LLMTS.

大模型医疗时间序列不规则数据编码器设计

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。