解决多模态时间序列缺失问题,提升模型鲁棒性。
TRACE: A Temporal Conditional Estimation for Multimodal Time Series Foundation Models

- 基于辅助模态条件推断缺失目标模态,建模跨模态依赖。
- 在医疗与情感分析数据集上显著优于现有方法。
- 适合处理不规则采样和严重模态缺失的场景。
时间序列基础模型(TS-FMs)旨在学习可泛化的时序表征,以适应多种下游任务。在真实多模态场景中,时间序列常面临时序错位与部分模态缺失问题,即不同模态以异质时间尺度观测或部分缺失。现有方法多依赖简单插补或掩码策略,未能充分考虑跨模态依赖,常导致表征错位或退化。本文提出TRACE,一种面向多模态时间序列基础模型在缺失与不规则采样下的条件估计范式,使不完整的目标模态可系统性地从可用的辅助模态中推断。我们在涵盖医疗与情感计算的多个基准上评估TRACE,包括MIMIC-IV临床数据集及CMU-MOSI和CMU-MOSEI多模态情感分析基准。在多种下游预测任务与缺失模态设置下,TRACE始终优于现有融合方法,展现出对严重模态缺失更强的鲁棒性与更可靠的跨模态表征。
原文摘要 · Abstract (English)
Time series foundation models (TS-FMs) aim to learn generalizable temporal representations that can be adapted to a wide range of downstream tasks. In real-world multimodal settings, time series are frequently affected by temporal misalignment and partial modality missingness, where different modalities are observed at heterogeneous time scales or are partially absent. Existing approaches typically rely on naive imputation or masking strategies, which fail to account for cross-modal dependencies and often lead to misaligned or degraded representations. We propose TRACE, a conditional estimation paradigm for multimodal time series foundation model pipelines under missingness and irregular sampling, allowing incomplete target modalities to be systematically inferred from available auxiliary modalities. We evaluate TRACE on diverse multimodal benchmarks spanning healthcare and affective computing, including the MIMIC-IV clinical dataset and the CMU-MOSI and CMU-MOSEI benchmarks for multimodal sentiment analysis. Across a range of downstream prediction tasks and missing-modality settings, TRACE consistently outperforms prior multimodal fusion approaches, demonstrating improved robustness to severe modality missingness and more reliable cross-modal representations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。