用高效微调技术提升医疗时间序列模型性能，仅调2400参数超顶尖模型
Beyond LoRA: Exploring Efficient Fine-Tuning Techniques for Time Series Foundational Models
- 选用四种参数高效微调方法，聚焦重症患者生命体征预测
- 傅里叶微调在小模型上仅用2400参数超越现有最佳结果
- 适合资源受限场景下医疗时序数据的精准建模与快速部署
时间序列基础模型（TSFMs）近年来因其在零售、金融、交通等领域的复杂大规模数据建模能力而受到关注。然而，在医疗等敏感领域，由于缺乏公开可用的专用数据集，其在特定任务上的微调仍面临挑战。本文探讨了参数高效微调（PEFT）技术在医疗场景中的应用，特别是针对败血症患者的重症监护室生命体征预测。我们在多个配置的Chronos TSFM上评估了两种选择性方法（BitFit、LayerNorm Tuning）和两种加性方法（VeRA、FourierFT），结果表明部分PEFT方法在参数效率和领域适应性方面优于LoRA，实现了重症监护生命体征预测的最新技术水平。值得注意的是，将FourierFT应用于Chronos（Tiny）模型，仅微调2,400个参数，便超过了现有最佳模型，而基准模型需调整700,000个参数。
原文摘要 · Abstract (English)
Time Series Foundation Models (TSFMs) have recently garnered attention for their ability to model complex, large-scale time series data across domains such as retail, finance, and transportation. However, their application to sensitive, domain-specific fields like healthcare remains challenging, primarily due to the difficulty of fine-tuning these models for specialized, out-of-domain tasks with scarce publicly available datasets. In this work, we explore the use of Parameter-Efficient Fine-Tuning (PEFT) techniques to address these limitations, focusing on healthcare applications, particularly ICU vitals forecasting for sepsis patients. We introduce and evaluate two selective (BitFit and LayerNorm Tuning) and two additive (VeRA and FourierFT) PEFT techniques on multiple configurations of the Chronos TSFM for forecasting vital signs of sepsis patients. Our comparative analysis demonstrates that some of these PEFT methods outperform LoRA in terms of parameter efficiency and domain adaptation, establishing state-of-the-art (SOTA) results in ICU vital forecasting tasks. Interestingly, FourierFT applied to the Chronos (Tiny) variant surpasses the SOTA model while fine-tuning only 2,400 parameters compared to the 700K parameters of the benchmark.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。