arXiv:2603.29974cs.LG2026-03

用少量数据精准预测空气质量,跨区域通用性强。

Meteorology-Driven GPT4AP: A Multi-Task Forecasting LLM for Atmospheric Air Pollution in Data-Scarce Settings

  • 基于GPT-2改进,只训练位置和输出模块,参数少效率高。
  • 仅用10%数据时平均误差0.442,零样本跨站点迁移误差0.403。
  • 适合观测数据少的地区,也适用于数据丰富的长期预测场景。

准确预测空气污染对环境监测和政策制定至关重要,但数据驱动模型在观测稀疏区域常面临泛化能力不足的问题。本文提出基于预训练GPT-2骨干网络的气象驱动多任务预测框架GPT4AP,采用高斯秩稳定低秩适配(rsLoRA)技术,冻结自注意力与前馈层,仅微调轻量级位置和输出模块,显著减少可训练参数。在六个真实空气质量管理数据集上评估,涵盖少样本、零样本及长期预测场景。少样本设置下使用10%训练数据,平均MSE/MAE为0.686/0.442,优于DLinear(0.728/0.530)和ETSformer(0.734/0.505)。零样本跨站迁移中平均MSE/MAE达0.529/0.403,显示更强泛化能力。全数据长周期预测中平均MAE为0.429,表现与专用时序模型相当。结果表明,GPT4AP在低监督和领域偏移条件下仍具鲁棒性,且在数据充足时保持竞争力。

原文摘要 · Abstract (English)

Accurate forecasting of air pollution is important for environmental monitoring and policy support, yet data-driven models often suffer from limited generalization in regions with sparse observations. This paper presents Meteorology-Driven GPT for Air Pollution (GPT4AP), a parameter-efficient multi-task forecasting framework based on a pre-trained GPT-2 backbone and Gaussian rank-stabilized low-rank adaptation (rsLoRA). The model freezes the self-attention and feed-forward layers and adapts lightweight positional and output modules, substantially reducing the number of trainable parameters. GPT4AP is evaluated on six real-world air quality monitoring datasets under few-shot, zero-shot, and long-term forecasting settings. In the few-shot regime using 10% of the training data, GPT4AP achieves an average MSE/MAE of 0.686/0.442, outperforming DLinear (0.728/0.530) and ETSformer (0.734/0.505). In zero-shot cross-station transfer, the proposed model attains an average MSE/MAE of 0.529/0.403, demonstrating improved generalization compared with existing baselines. In long-term forecasting with full training data, GPT4AP remains competitive, achieving an average MAE of 0.429, while specialized time-series models show slightly lower errors. These results indicate that GPT4AP provides a data-efficient forecasting approach that performs robustly under limited supervision and domain shift, while maintaining competitive accuracy in data-rich settings.

空气污染少样本多任务GPT

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。