统一扩散模型融合文本与时间信息,提升多模态时序预测精度
UniDiff: A Unified Diffusion Framework for Multimodal Time Series Forecasting
- 将时序数据分块编码,用轻量MLP映射到嵌入空间
- 单步交叉注意力融合时间戳结构与文本语义,实现跨模态高效交互
- 新设计无分类器引导机制,可独立控制文本与时间的引导强度
随着多模态数据在现实应用中日益增长,如何利用文本、时间戳等异构信息进行准确的时序预测成为关键挑战。尽管扩散模型在生成任务中表现优异,但其在时序预测中的应用仍主要局限于单一数值序列,忽视了复杂异构数据中丰富的跨模态信号。为此,我们提出UniDiff,一个面向多模态时序预测的统一扩散框架。该框架首先将时序数据分块并用轻量MLP映射为嵌入向量,保留局部时间动态。核心是统一且并行的融合模块,通过单个交叉注意力机制自适应地加权和整合时间戳结构信息与文本语义上下文,实现模态间灵活高效的交互。此外,我们引入一种新型无分类器引导机制,适用于多源条件输入,可在推理阶段独立调节文本与时间信息的引导强度,显著提升模型鲁棒性。在八个领域的真实世界基准数据集上进行的大量实验表明,所提出的UniDiff模型达到当前最优性能。
原文摘要 · Abstract (English)
As multimodal data proliferates across diverse real-world applications, leveraging heterogeneous information such as texts and timestamps for accurate time series forecasting (TSF) has become a critical challenge. While diffusion models demonstrate exceptional performance in generation tasks, their application to TSF remains largely confined to modeling single-modality numerical sequences, overlooking the abundant cross-modal signals inherent in complex heterogeneous data. To address this gap, we propose UniDiff, a unified diffusion framework for multimodal time series forecasting. To process the numerical sequence, our framework first tokenizes the time series into patches, preserving local temporal dynamics by mapping each patch to an embedding space via a lightweight MLP. At its core lies a unified and parallel fusion module, where a single cross-attention mechanism adaptively weighs and integrates structural information from timestamps and semantic context from texts in one step, enabling a flexible and efficient interplay between modalities. Furthermore, we introduce a novel classifier-free guidance mechanism designed for multi-source conditioning, allowing for decoupled control over the guidance strength of textual and temporal information during inference, which significantly enhances model robustness. Extensive experiments on real-world benchmark datasets across eight domains demonstrate that the proposed UniDiff model achieves state-of-the-art performance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。