arXiv:2506.01364cs.LGcs.AI2025-06综述被引 31

系统梳理时空基础模型的完整训练流程,帮研究者快速上手。

Unraveling Spatio-Temporal Foundation Models via the Pipeline Lens: A Comprehensive Review

  • 从数据到训练,按流程解析时空基础模型设计
  • 提出新分类法,按数据源和依赖关系划分方法
  • 适合想入门或构建时空模型的研究者参考

时空深度学习模型旨在利用此类数据中的有用模式以支持预测等任务。然而,以往针对特定任务设计的深度学习模型通常需为每个应用场景单独训练,导致计算和存储成本增加。为此,时空基础模型应运而生,提供统一框架以解决多种时空任务。这些模型通过学习时空数据中的通用知识或迁移预训练语言模型的能力取得显著成功。尽管此前综述分别探讨了时空数据与方法,却未全面考察基础模型的设计、选择、预训练与适配过程,致使整体流程仍不清晰。为弥补这一空白,本文创新性地从流程视角全面回顾现有时空基础模型。流程始于不同类型的时空数据介绍,继而详述数据预处理与嵌入技术;随后提出新的数据属性分类法,依据数据来源与依赖关系对现有方法进行划分,助力研究人员高效设计与选型。在此基础上,进一步阐述原始模型的训练目标及迁移模型的适配技术。整体而言,本综述提供清晰结构化的流程,揭示时空基础模型核心要素间的关联,指导研究者快速入门。此外,还引入多目标训练等新兴机遇。

原文摘要 · Abstract (English)

Spatio-temporal deep learning models aims to utilize useful patterns in such data to support tasks like prediction. However, previous deep learning models designed for specific tasks typically require separate training for each use case, leading to increased computational and storage costs. To address this issue, spatio-temporal foundation models have emerged, offering a unified framework capable of solving multiple spatio-temporal tasks. These foundation models achieve remarkable success by learning general knowledge with spatio-temporal data or transferring the general capabilities of pre-trained language models. While previous surveys have explored spatio-temporal data and methodologies separately, they have ignored a comprehensive examination of how foundation models are designed, selected, pre-trained, and adapted. As a result, the overall pipeline for spatio-temporal foundation models remains unclear. To bridge this gap, we innovatively provide an up-to-date review of previous spatio-temporal foundation models from the pipeline perspective. The pipeline begins with an introduction to different types of spatio-temporal data, followed by details of data preprocessing and embedding techniques. The pipeline then presents a novel data property taxonomy to divide existing methods according to data sources and dependencies, providing efficient and effective model design and selection for researchers. On this basis, we further illustrate the training objectives of primitive models, as well as the adaptation techniques of transferred models. Overall, our survey provides a clear and structured pipeline to understand the connection between core elements of spatio-temporal foundation models while guiding researchers to get started quickly. Additionally, we introduce emerging opportunities such as multi-objective training in the field of spatio-temporal foundation models.

时空模型基础模型综述

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。