arXiv:2503.17994cs.CLcs.AI2025-03被引 1

用大模型指导时空序列预测的网络结构搜索,提升效率与效果。

Instructing the Architecture Search for Spatial-temporal Sequence Forecasting with LLM

  • 通过多层级增强机制,让大模型充当架构搜索的导师。
  • 在多个数据集上达到领先性能,搜索速度比传统方法快数倍。
  • 适合需要高效自动化设计模型的研究者与工业应用。

时空序列预测(STSF)是长期存在的研究问题,具有广泛的实际应用。神经架构搜索(NAS)通过自动化神经网络设计,在解决STSF问题上已展现出有效性。然而,现有的STSF-NAS方法依赖耗时的数据驱动方式生成架构,严重限制了其对背景知识的利用和复杂搜索路径的探索能力。大语言模型(LLM)具备基于丰富内部世界知识进行决策的能力,但其如何助力STSF-NAS仍待探索。本文提出一种基于LLM的新型STSF-NAS方法。我们不直接由LLM生成架构,而是通过多层次增强机制激发其潜力:在步骤层面,将生成任务分解为决策步骤,借助强大提示工程,使LLM基于内部知识担任架构搜索的指导者;在实例层面,采用一步调优框架快速评估架构实例,并利用记忆库累积知识以提升LLM的搜索能力;在任务层面,提出两阶段架构搜索策略,平衡探索与优化阶段,降低陷入局部最优的可能性。大量实验结果表明,该方法在保持竞争力性能的同时,相比现有STSF-NAS方法显著提升效率。

原文摘要 · Abstract (English)

Spatial-temporal sequence forecasting (STSF) is a long-standing research problem with widespread real-world applications. Neural architecture search (NAS), which automates the neural network design, has been shown effective in tackling the STSF problem. However, the existing NAS methods for STSF focus on generating architectures in a time-consuming data-driven fashion, which heavily limits their ability to use background knowledge and explore the complicated search trajectory. Large language models (LLMs) have shown remarkable ability in decision-making with comprehensive internal world knowledge, but how it could benefit NAS for STSF remains unexplored. In this paper, we propose a novel NAS method for STSF based on LLM. Instead of directly generate architectures with LLM, We inspire the LLM's capability with a multi-level enhancement mechanism. Specifically, on the step-level, we decompose the generation task into decision steps with powerful prompt engineering and inspire LLM to serve as instructor for architecture search based on its internal knowledge. On the instance-level, we utilize a one-step tuning framework to quickly evaluate the architecture instance and a memory bank to cumulate knowledge to improve LLM's search ability. On the task-level, we propose a two-stage architecture search, balancing the exploration stage and optimization stage, to reduce the possibility of being trapped in local optima. Extensive experimental results demonstrate that our method can achieve competitive effectiveness with superior efficiency against existing NAS methods for STSF.

架构搜索大模型时空预测

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。