arXiv:2504.14126cs.AIcs.CL2025-04被引 11

用大模型优化粒子群算法,加速深度学习超参搜索。

Large Language Model Enhanced Particle Swarm Optimization for Hyperparameter Tuning for Deep Learning Models

  • 用大模型替代低效粒子位置,提升搜索效率
  • 减少20%至60%计算量,保持精度不变
  • 适合需要快速调参的科研与工程场景

确定深度学习模型的理想结构(如层数、神经元数)是一项困难且资源密集的任务,通常依赖人工调参或计算成本高昂的优化方法。尽管粒子群优化(PSO)和大语言模型(LLMs)分别应用于优化与深度学习,但两者结合以提升数值优化收敛性仍少有研究。本文提出一种融合LLM的PSO方法,通过ChatGPT-3.5和Llama3为劣质粒子位置提供最优建议,替代低效搜索,加快探索速度。在三个场景中验证:(1) Rastrigin函数优化,(2) LSTM用于时间序列回归,(3) CNN用于材料分类。结果表明,该方法显著提升收敛速度并降低计算成本,相较传统PSO,计算复杂度降低20%至60%。在回归任务中,Llama3减少20%-40%模型调用;ChatGPT-3.5在回归与分类任务中分别减少60%模型调用,且保持准确率与误差率不变。该方法为深度学习模型优化提供高效解决方案。

原文摘要 · Abstract (English)

Determining the ideal architecture for deep learning models, such as the number of layers and neurons, is a difficult and resource-intensive process that frequently relies on human tuning or computationally costly optimization approaches. While Particle Swarm Optimization (PSO) and Large Language Models (LLMs) have been individually applied in optimization and deep learning, their combined use for enhancing convergence in numerical optimization tasks remains underexplored. Our work addresses this gap by integrating LLMs into PSO to reduce model evaluations and improve convergence for deep learning hyperparameter tuning. The proposed LLM-enhanced PSO method addresses the difficulties of efficiency and convergence by using LLMs (particularly ChatGPT-3.5 and Llama3) to improve PSO performance, allowing for faster achievement of target objectives. Our method speeds up search space exploration by substituting underperforming particle placements with best suggestions offered by LLMs. Comprehensive experiments across three scenarios -- (1) optimizing the Rastrigin function, (2) using Long Short-Term Memory (LSTM) networks for time series regression, and (3) using Convolutional Neural Networks (CNNs) for material classification -- show that the method significantly improves convergence rates and lowers computational costs. Depending on the application, computational complexity is lowered by 20% to 60% compared to traditional PSO methods. Llama3 achieved a 20% to 40% reduction in model calls for regression tasks, whereas ChatGPT-3.5 reduced model calls by 60% for both regression and classification tasks, all while preserving accuracy and error rates. This groundbreaking methodology offers a very efficient and effective solution for optimizing deep learning models, leading to substantial computational performance improvements across a wide range of applications.

超参优化大模型粒子群优化深度学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。