arXiv:2411.18731cs.AIcs.SE2024-11被引 12

用大模型生成的LSTM代码能有效预测时间序列,效果接近人工优化。

The Performance of the LSTM-based Code Generated by Large Language Models (LLMs) in Forecasting Time Series Data

  • 通过控制提示词的清晰度、目标等四方面,测试大模型生成LSTM代码的效果。
  • 生成代码的预测性能接近人工优化模型,且ChatGPT表现最优。
  • 温度参数影响生成质量,适合想快速构建预测模型的数据分析师。

本文研究了主流大语言模型(如ChatGPT、PaLM、LLama、Falcon)在生成用于时间序列分析的深度学习模型方面的表现。研究通过受控实验,考察了提示词在清晰性、目标明确性、上下文信息和格式风格四个维度上的敏感性。结果表明,利用大模型可为每个数据集生成可执行的深度学习代码,其预测性能与人工精心设计和优化的LSTM模型相当。其中,ChatGPT生成的模型准确率更高,且模型质量随生成时使用的温度参数变化而变化。该研究对缺乏深度学习经验的数据分析师具有实用价值,有助于借助生成式AI快速构建具备合理性能的预测模型。

原文摘要 · Abstract (English)

As an intriguing case is the goodness of the machine and deep learning models generated by these LLMs in conducting automated scientific data analysis, where a data analyst may not have enough expertise in manually coding and optimizing complex deep learning models and codes and thus may opt to leverage LLMs to generate the required models. This paper investigates and compares the performance of the mainstream LLMs, such as ChatGPT, PaLM, LLama, and Falcon, in generating deep learning models for analyzing time series data, an important and popular data type with its prevalent applications in many application domains including financial and stock market. This research conducts a set of controlled experiments where the prompts for generating deep learning-based models are controlled with respect to sensitivity levels of four criteria including 1) Clarify and Specificity, 2) Objective and Intent, 3) Contextual Information, and 4) Format and Style. While the results are relatively mix, we observe some distinct patterns. We notice that using LLMs, we are able to generate deep learning-based models with executable codes for each dataset seperatly whose performance are comparable with the manually crafted and optimized LSTM models for predicting the whole time series dataset. We also noticed that ChatGPT outperforms the other LLMs in generating more accurate models. Furthermore, we observed that the goodness of the generated models vary with respect to the ``temperature'' parameter used in configuring LLMS. The results can be beneficial for data analysts and practitioners who would like to leverage generative AIs to produce good prediction models with acceptable goodness.

时间序列大模型生成LSTM自动化建模

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。