arXiv:2508.05433cs.LGcs.NE2025-08被引 6

用大模型+进化搜索生成可解释的控制策略,让算法更透明好懂。

Multimodal LLM-assisted Evolutionary Search for Programmatic Control Policies

  • 用多模态大模型生成程序化控制逻辑,结合进化搜索自动优化。
  • 在两个标准任务上性能接近PPO,且策略逻辑完全可追溯。
  • 无需预设语言,支持跨任务迁移,适合需要可解释性的场景。

深度强化学习在控制任务中表现优异,但其策略以黑箱神经网络形式存在,难以理解、验证和调试,影响信任度并阻碍实际应用。本文提出一种名为多模态大语言模型辅助进化搜索(MLES)的新方法,用于发现程序化控制策略。MLES利用多模态大语言模型作为程序化策略生成器,结合进化搜索实现自动化策略生成,并在过程中融入视觉反馈驱动的行为分析,识别失败模式并指导针对性改进,从而提升策略发现效率,生成可适应、与人类对齐的策略。实验表明,MLES在两个标准控制任务上的性能可媲美近端策略优化(PPO),同时提供透明的控制逻辑和可追溯的设计过程。该方法克服了预定义领域特定语言的局限,促进知识迁移与复用,具备跨任务可扩展性,展现出构建透明、可验证控制策略的新范式前景。代码已公开于 https://github.com/QingL2000/MLES。

原文摘要 · Abstract (English)

Deep reinforcement learning has achieved impressive success in control tasks. However, its policies, represented as opaque neural networks, are often difficult for humans to understand, verify, and debug, which undermines trust and hinders real-world deployment. This work addresses this challenge by introducing a novel approach for programmatic control policy discovery, called Multimodal Large Language Model-assisted Evolutionary Search (MLES). MLES utilizes multimodal large language models as programmatic policy generators, combining them with evolutionary search to automate policy generation. It integrates visual feedback-driven behavior analysis within the policy generation process to identify failure patterns and guide targeted improvements, thereby enhancing policy discovery efficiency and producing adaptable, human-aligned policies. Experimental results demonstrate that MLES achieves performance comparable to Proximal Policy Optimization (PPO) across two standard control tasks while providing transparent control logic and traceable design processes. This approach also overcomes the limitations of predefined domain-specific languages, facilitates knowledge transfer and reuse, and is scalable across various tasks, showing promise as a new paradigm for developing transparent and verifiable control policies. Code is publicly available at https://github.com/QingL2000/MLES.

强化学习可解释性程序生成大模型应用

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。