首个融合新闻与电价的多模态预测基准,检验大模型在真实电力市场中的表现。
NSW-EPNews: A News-Augmented Benchmark for Electricity Price Forecasting with LLMs
- 构建包含17.5万条电价、天气和新闻摘要的多模态数据集,支持48步长预测
- 传统模型加入新闻信息提升有限,大模型虽有小幅改进但常产生虚构价格序列
- 适合研究多模态推理、能源预测或评估大模型可靠性的人群使用
电力价格预测是现代能源管理系统的关键环节,但现有方法主要依赖历史数值数据,忽略同期文本信息。本文提出 NSW-EPNews,首个联合评估时间序列模型与大语言模型(LLMs)的真实电力价格预测基准。数据集包含澳大利亚新南威尔士州2015–2024年超过17.5万条半小时级现货电价、每日气温读数,以及来自WattClarity的精选市场新闻摘要。任务设定为48步长预测,输入包括滞后电价、向量化新闻与天气特征(用于经典模型),以及提示工程生成的结构化上下文(用于LLMs)。共生成3600个用于LLM评估的多模态提示-输出对。实验表明,传统统计与机器学习模型从新闻特征中获益甚微;而先进大模型如GPT-4o与Gemini 1.5 Pro虽有小幅性能提升,却频繁出现幻觉,如虚构或格式错误的价格序列。NSW-EPNews为多模态场景下的可信数值推理提供了严格测试平台,揭示当前大模型能力与高风险能源预测需求之间的显著差距。
原文摘要 · Abstract (English)
Electricity price forecasting is a critical component of modern energy-management systems, yet existing approaches heavily rely on numerical histories and ignore contemporaneous textual signals. We introduce NSW-EPNews, the first benchmark that jointly evaluates time-series models and large language models (LLMs) on real-world electricity-price prediction. The dataset includes over 175,000 half-hourly spot prices from New South Wales, Australia (2015-2024), daily temperature readings, and curated market-news summaries from WattClarity. We frame the task as 48-step-ahead forecasting, using multimodal input, including lagged prices, vectorized news and weather features for classical models, and prompt-engineered structured contexts for LLMs. Our datasets yields 3.6k multimodal prompt-output pairs for LLM evaluation using specific templates. Through compresive benchmark design, we identify that for traditional statistical and machine learning models, the benefits gain is marginal from news feature. For state-of-the-art LLMs, such as GPT-4o and Gemini 1.5 Pro, we observe modest performance increase while it also produce frequent hallucinations such as fabricated and malformed price sequences. NSW-EPNews provides a rigorous testbed for evaluating grounded numerical reasoning in multimodal settings, and highlights a critical gap between current LLM capabilities and the demands of high-stakes energy forecasting.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。