用自动化方法构建气候问答数据集,训练出更懂气候变化的AI助手。
ClimateChat: Designing Data and Methods for Instruction Tuning LLMs to Answer Climate Change Queries
- 通过文档事实+网络爬虫+种子指令生成多样化问答数据
- 新数据集使模型在气候问答任务上表现显著提升
- 适合气候研究者和政策制定者快速获取科学信息
随着全球气候变化问题日益严峻,气候科学领域的研究需求持续增长。以大语言模型(LLMs)为代表的自然语言处理技术已被广泛应用于气候相关研究,为决策者和公众提供关键信息支持。尽管已有研究通过构建气候相关指令数据并微调大模型提升了性能,但当前仍缺乏高效生成大规模高精度指令数据的方法,制约了气候专用大模型的发展。本研究提出一种自动化指令数据构建方法:基于文档中的事实与背景知识生成指令,并结合网络爬虫和种子指令收集增强数据多样性。据此构建了名为ClimateChat-Corpus的气候指令数据集,并用于微调开源大模型,得到ClimateChat模型。评估结果表明,ClimateChat在气候问答任务上表现显著提升。此外,我们对比了不同基础模型与指令数据对性能的影响,证明其可适应多种气候科学发现任务,强调选择合适基础模型的重要性。本研究为构建气候指令数据和训练专用大模型提供了重要参考与实证支持。
原文摘要 · Abstract (English)
As the issue of global climate change becomes increasingly severe, the demand for research in climate science continues to grow. Natural language processing technologies, represented by Large Language Models (LLMs), have been widely applied to climate change-specific research, providing essential information support for decision-makers and the public. Some studies have improved model performance on relevant tasks by constructing climate change-related instruction data and instruction-tuning LLMs. However, current research remains inadequate in efficiently producing large volumes of high-precision instruction data for climate change, which limits further development of climate change LLMs. This study introduces an automated method for constructing instruction data. The method generates instructions using facts and background knowledge from documents and enhances the diversity of the instruction data through web scraping and the collection of seed instructions. Using this method, we constructed a climate change instruction dataset, named ClimateChat-Corpus, which was used to fine-tune open-source LLMs, resulting in an LLM named ClimateChat. Evaluation results show that ClimateChat significantly improves performance on climate change question-and-answer tasks. Additionally, we evaluated the impact of different base models and instruction data on LLM performance and demonstrated its capability to adapt to a wide range of climate change scientific discovery tasks, emphasizing the importance of selecting an appropriate base model for instruction tuning. This research provides valuable references and empirical support for constructing climate change instruction data and training climate change-specific LLMs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。