让大模型通过迭代优化直接生成机器人控制策略
Sensory-Motor Control with Large Language Models via Iterative Policy Refinement
- 用文本描述生成初始控制策略，再通过反馈迭代优化
- 在Gymnasium和MuJoCo任务中成功找到近优解
- 适合想用大模型做机器人控制的开发者
我们提出一种方法，使大语言模型（LLMs）能够通过生成控制策略来操控具身智能体，该策略直接将连续观测向量映射为连续动作向量。初始阶段，LLM基于对智能体、环境及目标的文本描述生成控制策略。随后，通过一个学习过程不断迭代优化：在评估过程中收集感官-运动数据，并结合性能反馈，反复提示LLM改进当前策略。该方法在Gymnasium经典控制任务和MuJoCo的倒立摆任务上得到验证。即使使用相对较小的模型如GPT-oss:120b和Qwen2.5:72b，也多数情况下成功找到最优或近优解。其有效性源于将符号推理知识与智能体交互中获取的非符号感官-运动数据相结合。
原文摘要 · Abstract (English)
We propose a method that enables large language models (LLMs) to control embodied agents through the generation of control policies that directly map continuous observation vectors to continuous action vectors. At the outset, the LLMs generate a control strategy based on a textual description of the agent, its environment, and the intended goal. This strategy is then iteratively refined through a learning process in which the LLMs are repeatedly prompted to improve the current strategy, using performance feedback and sensory-motor data collected during its evaluation. The method is validated on classic control tasks from the Gymnasium library and the inverted pendulum task from the MuJoCo library. The approach proves effective with relatively compact models such as GPT-oss:120b and Qwen2.5:72b. In most cases, it successfully identifies optimal or near-optimal solutions by integrating symbolic knowledge derived through reasoning with sub-symbolic sensory-motor data gathered as the agent interacts with its environment.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。