arXiv:2602.14017cs.LG2026-02

评测大模型在气候服务中的决策能力,填补科学预报到实际应用的最后差距。

S2SServiceBench: A Multimodal Benchmark for Last-Mile S2S Climate Services

  • 构建多模态基准,模拟真实气候服务场景下的决策任务。
  • 包含500+任务、10类服务产品,覆盖农业到航运六大领域。
  • 揭示模型在理解不确定性与生成可执行决策上的关键短板。

次季节至季节(S2S)预报对气候韧性与可持续发展具有重要意义,但存在‘最后一公里’瓶颈:如何将科学预报转化为可信、可操作的气候服务,需在不确定条件下具备可靠的多模态理解和面向决策的推理能力。尽管多模态大语言模型(MLLMs)及其代理范式在各类流程中进展迅速,其能否在不确定性下从实际服务产品中可靠生成决策成果仍不明确。本文提出S2SServiceBench,一个基于实际气候服务系统的多模态基准,用于评估该能力。该基准涵盖10类服务产品,共约150+位专家精选案例,覆盖农业、灾害、能源、金融、健康和航运六大应用领域。每项案例按三个服务层级实例化,共生成约500个任务和1000+个评估项,应用于气候韧性与可持续性场景。通过该基准,我们对主流MLLMs与代理进行评测,分析其在不同产品与服务层级的表现,揭示出在可行动信号理解、将不确定性转化为可执行交付、以及动态灾害下的稳定证据驱动分析与规划方面存在的持续挑战,并为构建未来气候服务代理提供可操作指导。

原文摘要 · Abstract (English)

Subseasonal-to-seasonal (S2S) forecasts play an essential role in providing a decision-critical weeks-to-months planning window for climate resilience and sustainability, yet a growing bottleneck is the last-mile gap: translating scientific forecasts into trusted, actionable climate services, requiring reliable multimodal understanding and decision-facing reasoning under uncertainty. Meanwhile, multimodal large language models (MLLMs) and corresponding agentic paradigms have made rapid progress in supporting various workflows, but it remains unclear whether they can reliably generate decision-making deliverables from operational service products (e.g., actionable signal comprehension, decision-making handoff, and decision analysis & planning) under uncertainty. We introduce S2SServiceBench, a multimodal benchmark for last-mile S2S climate services curated from an operational climate-service system to evaluate this capability. S2SServiceBenchcovers 10 service products with about 150+ expert-selected cases in total, spanning six application domains - Agriculture, Disasters, Energy, Finance, Health, and Shipping. Each case is instantiated at three service levels, yielding around 500 tasks and 1,000+ evaluation items across climate resilience and sustainability applications. Using S2SServiceBench, we benchmark state-of-the-art MLLMs and agents, and analyze performance across products and service levels, revealing persistent challenges in S2S service plot understanding and reasoning - namely, actionable signal comprehension, operationalizing uncertainty into executable handoffs, and stable, evidence-grounded analysis and planning for dynamic hazards-while offering actionable guidance for building future climate-service agents.

气候服务多模态大模型决策支持

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。