用强化学习让多模态大模型学会时间序列推理,表现超越主流模型。
TimeMaster: Training Time-Series Multimodal LLMs to Reason via Reinforcement Learning
- 基于强化学习设计三段式输出结构,引导模型生成可解释的推理过程。
- 在六个真实任务上超越GPT-4o 7.3%,比传统时序模型高14.6%。
- 适合需要可解释时序分析的金融、医疗等专业场景使用者。
时间序列推理在多模态大语言模型中仍面临动态时间模式、语义模糊和缺乏时间先验等挑战。本文提出TimeMaster,一种基于强化学习的方法,使时间序列多模态大模型能直接对可视化时序输入和任务提示进行结构化、可解释的推理。TimeMaster采用三部分输出格式:推理、分类与领域特定扩展,并通过复合奖励函数优化,该函数兼顾格式遵循性、预测准确率和开放性洞察质量。训练采用两阶段流程:先进行监督微调(SFT)初始化,再使用逐标记级别的组相对策略优化(GRPO)实现稳定且目标明确的奖励驱动改进。在基于Qwen2.5-VL-3B-Instruct的TimerBed基准上评估,涵盖六项真实世界分类任务,TimeMaster达到领先性能,相比经典时序模型提升超14.6%,比少样本GPT-4o高7.3%。值得注意的是,TimeMaster不仅完成分类,还表现出专家级推理行为,生成上下文相关的解释并提供领域对齐见解。结果表明,奖励驱动的强化学习是将时间理解融入时间序列多模态大模型的可行且有前景路径。
原文摘要 · Abstract (English)
Time-series reasoning remains a significant challenge in multimodal large language models (MLLMs) due to the dynamic temporal patterns, ambiguous semantics, and lack of temporal priors. In this work, we introduce TimeMaster, a reinforcement learning (RL)-based method that enables time-series MLLMs to perform structured, interpretable reasoning directly over visualized time-series inputs and task prompts. TimeMaster adopts a three-part structured output format, reasoning, classification, and domain-specific extension, and is optimized via a composite reward function that aligns format adherence, prediction accuracy, and open-ended insight quality. The model is trained using a two-stage pipeline: we first apply supervised fine-tuning (SFT) to establish a good initialization, followed by Group Relative Policy Optimization (GRPO) at the token level to enable stable and targeted reward-driven improvement in time-series reasoning. We evaluate TimeMaster on the TimerBed benchmark across six real-world classification tasks based on Qwen2.5-VL-3B-Instruct. TimeMaster achieves state-of-the-art performance, outperforming both classical time-series models and few-shot GPT-4o by over 14.6% and 7.3% performance gain, respectively. Notably, TimeMaster goes beyond time-series classification: it also exhibits expert-like reasoning behavior, generates context-aware explanations, and delivers domain-aligned insights. Our results highlight that reward-driven RL can be a scalable and promising path toward integrating temporal understanding into time-series MLLMs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。