用大模型驱动搜索多源强化学习状态编码器，更高效找到好方案。
LLM-Driven Composite Neural Architecture Search for Multi-Source RL State Encoding
- 大模型当设计助手，结合中间输出信号优化编码结构
- 在交通控制任务中用更少评估次数达到更好性能
- 适合需要融合图像、文本等多源信息的RL研究者
为融合传感器数据、时间序列、图像观测和文本指令等多种信息源的强化学习（RL）设计状态编码器仍缺乏系统方法，通常依赖人工设计。本文将该问题形式化为复合神经架构搜索（NAS）任务，联合优化各源专用模块与融合模块。现有NAS方法忽略模块中间输出中的表征质量等有用辅助信息，导致多源RL场景下样本效率低下。为此，我们提出一种大语言模型（LLM）驱动的NAS流程，让LLM作为神经架构设计代理，利用语言模型先验和中间输出信号，引导高效率搜索高性能复合状态编码器。在混合自主交通控制任务中，该方法以更少的候选评估次数获得优于传统NAS基线和基于LLM的GENIUS框架的性能。
原文摘要 · Abstract (English)
Designing state encoders for reinforcement learning (RL) with multiple information sources -- such as sensor measurements, time-series signals, image observations, and textual instructions -- remains underexplored and often requires manual design. We formalize this challenge as a problem of composite neural architecture search (NAS), where multiple source-specific modules and a fusion module are jointly optimized. Existing NAS methods overlook useful side information from the intermediate outputs of these modules -- such as their representation quality -- limiting sample efficiency in multi-source RL settings. To address this, we propose an LLM-driven NAS pipeline in which the LLM serves as a neural architecture design agent, leveraging language-model priors and intermediate-output signals to guide sample-efficient search for high-performing composite state encoders. On a mixed-autonomy traffic control task, our approach discovers higher-performing architectures with fewer candidate evaluations than traditional NAS baselines and the LLM-based GENIUS framework.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。