arXiv:2506.09114cs.LG2025-06NeurIPS被引 8

让时间序列与文本对齐,提升多模态检索与预测效果

TRACE: Grounding Time Series in Context for Multimodal Embedding and Retrieval

  • 通过文本上下文锚定时间序列嵌入,实现通道级精准对齐
  • 在多领域数据上显著提升预测准确率与可解释性
  • 既可作通用检索器,也可作为轻量级下游编码器

气象、医疗、能源等领域的动态数据日益普遍,亟需高效的时间序列解读与检索。这些数据天然依赖特定领域上下文(如临床记录或天气描述),跨模态检索不仅对下游任务至关重要,也助力构建基于检索增强生成(RAG)的时间序列基础模型。然而,现有方法普遍缺乏语义锚定,难以对齐异构模态,且处理多通道信号能力有限。为此,我们提出 TRACE,一种通用的多模态检索器,通过对齐的文本上下文来锚定时间序列嵌入。该方法支持细粒度通道级对齐,并采用硬负样本挖掘实现语义相关检索。它具备灵活的跨模态模式,包括 Text-to-Timeseries 与 Timeseries-to-Text,有效关联语言描述与复杂时序模式。通过检索语义相关的数据对,TRACE 为下游模型提供丰富上下文,提升预测准确率与可解释性。除静态检索外,TRACE 还可作为轻量级独立编码器,经少量任务微调即可优化上下文感知表示,同时保持强跨模态对齐。其在多个下游任务中达到当前最优性能。跨领域实验证明,它兼具强大编码能力与通用检索价值。

原文摘要 · Abstract (English)

The ubiquity of dynamic data in domains such as weather, healthcare, and energy underscores a growing need for effective interpretation and retrieval of time-series data. These data are inherently tied to domain-specific contexts, such as clinical notes or weather narratives, making cross-modal retrieval essential not only for downstream tasks but also for developing robust time-series foundation models by retrieval-augmented generation (RAG). Despite the increasing demand, time-series retrieval remains largely underexplored. Existing methods often lack semantic grounding, struggle to align heterogeneous modalities, and have limited capacity for handling multi-channel signals. To address this gap, we propose TRACE, a generic multimodal retriever that grounds time-series embeddings in aligned textual context. TRACE enables fine-grained channel-level alignment and employs hard negative mining to facilitate semantically meaningful retrieval. It supports flexible cross-modal retrieval modes, including Text-to-Timeseries and Timeseries-to-Text, effectively linking linguistic descriptions with complex temporal patterns. By retrieving semantically relevant pairs, TRACE enriches downstream models with informative context, leading to improved predictive accuracy and interpretability. Beyond a static retrieval engine, TRACE also serves as a powerful standalone encoder, with lightweight task-specific tuning that refines context-aware representations while maintaining strong cross-modal alignment. These representations achieve state-of-the-art performance on downstream forecasting and classification tasks. Extensive experiments across multiple domains highlight its dual utility, as both an effective encoder for downstream applications and a general-purpose retriever to enhance time-series models.

时间序列多模态检索RAG上下文对齐

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。