首个多模态气候事件预测数据集，连接气象数据与新闻文本
CLLMate: A Multimodal Benchmark for Weather and Climate Events Forecasting
- 用26,156篇新闻和ERA5数据构建多模态对齐数据集
- 23个主流多模态模型在该数据集上测试，揭示性能差异
- 适合研究气候智能生成、跨模态对齐的学者使用
天气与气候事件预测对减轻环境灾害、减少损失至关重要。然而现有研究多聚焦于温度等数值气象变量的预测，忽视将这些变量转化为可行动的事件描述与后果文本。为此，我们提出天气与气候事件预测（WCEF）新任务，利用数值气象栅格数据与文本事件数据预测天气与气候事件。由于多模态数据对齐困难及缺乏监督数据，该任务极具挑战。为此，我们构建了首个用于WCEF的多模态数据集CLLMate，包含26,156篇环境新闻文章与ERA5再分析数据的对齐样本。我们系统性地在CLLMate上评估了23个现有多模态大模型（包括闭源、开源及自研微调模型），实验揭示了现有模型的优势与局限，并验证了CLLMate在训练与评测WCEF任务中的价值。
原文摘要 · Abstract (English)
Forecasting weather and climate events is crucial for making appropriate measures to mitigate environmental hazards and minimize losses. However, existing environmental forecasting research focuses narrowly on predicting numerical meteorological variables (e.g., temperature), neglecting the translation of these variables into actionable textual narratives of events and their consequences. To bridge this gap, we proposed Weather and Climate Event Forecasting (WCEF), a new task that leverages numerical meteorological raster data and textual event data to predict weather and climate events. This task is challenging to accomplish due to difficulties in aligning multimodal data and the lack of supervised datasets. To address these challenges, we present CLLMate, the first multimodal dataset for WCEF, using 26,156 environmental news articles aligned with ERA5 reanalysis data. We systematically benchmark 23 existing MLLMs on CLLMate, including closed-source, open-source, and our fine-tuned models. Our experiments reveal the advantages and limitations of existing MLLMs and the value of CLLMate for the training and benchmarking of the WCEF task.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。