用大模型理解时间序列与自然语言的交互，提升多模态问答准确率。
ITFormer: Bridging Time Series and Natural Language for Multi-Modal QA with Large-Scale Multitask Dataset
- 将时间序列编码器与冻结大模型结合，实现跨模态特征对齐融合。
- 在新数据集上比基线提升显著，仅增加不足1%参数量。
- 适合做时间序列分析与智能对话系统的研究者参考。
时间序列数据在工业监控、医疗诊断和气候研究等众多领域至关重要。然而，如何有效将高维时序信号与自然语言整合，以支持动态交互任务仍是重大挑战。为此，我们提出时间序列问答（Time-Series QA）任务，并发布EngineMT-QA——首个大规模、多任务、时序-文本问答数据集，旨在捕捉时序信号与自然语言间的复杂互动。基于该数据集，我们提出指令式时间变换器（ITFormer），一种新型框架，将时间序列编码器与冻结的大语言模型（LLMs）相连接。ITFormer能有效提取、对齐并融合时序与文本特征，在仅增加少于1%可训练参数的情况下，显著提升问答准确率。通过兼顾计算效率与强大的跨模态建模能力，本工作建立了一种可扩展的时序数据与自然语言融合范式，为多模态AI的新研究与应用铺平道路。项目详情，包括数据集与代码，详见：https://pandalin98.github.io/itformer_site/
原文摘要 · Abstract (English)
Time-series data are critical in diverse applications, such as industrial monitoring, medical diagnostics, and climate research. However, effectively integrating these high-dimensional temporal signals with natural language for dynamic, interactive tasks remains a significant challenge. To address this, we introduce the Time-Series Question Answering (Time-Series QA) task and release EngineMT-QA, the first large-scale, multi-task, temporal-textual QA dataset designed to capture complex interactions between time-series signals and natural language. Building on this resource, we propose the Instruct Time Transformer (ITFormer), a novel framework that bridges time-series encoders with frozen large language models (LLMs). ITFormer effectively extracts, aligns, and fuses temporal and textual features, achieving a strong improvement in QA accuracy over strong baselines with fewer than 1\% additional trainable parameters. By combining computational efficiency with robust cross-modal modeling, our work establishes a adaptable paradigm for integrating temporal data with natural language, paving the way for new research and applications in multi-modal AI. More details about the project, including datasets and code, are available at: https://pandalin98.github.io/itformer_site/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。