arXiv:2504.08057cs.NEcs.AI2025-04中稿 · IEEE Transactions …被引 5

无需先验知识,自动构建多样解空间的优化新方法。

Vector Quantized-Elites: Unsupervised and Problem-Agnostic Quality-Diversity Optimization

  • 用向量量化VAE自动学习行为描述符,构建结构化解空间。
  • 在机械臂、机器人等任务中实现高质多样解,覆盖率达92%以上。
  • 适合无标签、未知任务场景,可推广至复杂现实问题。

质量-多样性算法通过寻找多样化且高性能的解,革新了优化范式。但传统方法如MAP-Elites依赖预定义的行为描述符和任务先验知识,限制了灵活性与适用性。本文提出向量量化精英(VQ-Elites),一种基于无监督学习自动生成结构化行为空间网格的新算法,无需任务特定知识。其核心是集成向量量化变分自编码器(VQ-VAE),动态学习行为描述符并生成有序行为空间,显著优于现有无监督方法。为提升性能,引入行为空间边界约束与协作机制,显著改善收敛速度与性能,并提出有效多样性比(Effective Diversity Ratio)和覆盖率得分(Coverage Diversity Score)以量化无监督环境下的实际多样性。在机械臂姿态到达、移动机器人空间覆盖及MiniGrid探索任务中验证,结果表明VQ-Elites能高效生成高质量、多样化的解,展现出强适应性、可扩展性、对超参数鲁棒性,以及向复杂未探索领域拓展的潜力。

原文摘要 · Abstract (English)

Quality-Diversity algorithms have transformed optimization by prioritizing the discovery of diverse, high-performing solutions over a single optimal result. However, traditional Quality-Diversity methods, such as MAP-Elites, rely heavily on predefined behavior descriptors and complete prior knowledge of the task to define the behavior space grid, limiting their flexibility and applicability. In this work, we introduce Vector Quantized-Elites (VQ-Elites), a novel Quality-Diversity algorithm that autonomously constructs a structured behavior space grid using unsupervised learning, eliminating the need for prior task-specific knowledge. At the core of VQ-Elites is the integration of Vector Quantized Variational Autoencoders, which enables the dynamic learning of behavior descriptors and the generation of a structured, rather than unstructured, behavior space grid -- a significant advancement over existing unsupervised Quality-Diversity approaches. This design establishes VQ-Elites as a flexible, robust, and task-agnostic optimization framework. To further enhance the performance of unsupervised Quality-Diversity algorithms, we introduce behavior space bounding and cooperation mechanisms, which significantly improve convergence and performance, as well as the Effective Diversity Ratio and Coverage Diversity Score, two novel metrics that quantify the actual diversity in the unsupervised setting. We validate VQ-Elites on robotic arm pose-reaching, mobile robot space-covering, and MiniGrid exploration tasks. The results demonstrate its ability to efficiently generate diverse, high-quality solutions, emphasizing its adaptability, scalability, robustness to hyperparameters, and potential to extend Quality-Diversity optimization to complex, previously inaccessible domains.

质量多样性无监督学习强化学习机器人

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。