通过增加行为多样性奖励，提升多智能体系统多目标优化效果。
Entropy-Augmented Multi-Objective Policy Optimization in Multiagent Systems
- 在评估中加入熵奖励，鼓励策略行为多样化
- 相比NSGA-II，帕累托解集超体积提升最高48%
- 适合需要探索多样行为策略的多智能体场景
在海洋或外星基地等场景中，自主智能体团队需协同行动以实现多个竞争目标。传统多目标进化算法如NSGA-II虽能优化目标空间多样性，却忽视行为空间多样性，可能导致过早收敛及策略同质化。为此，本文提出一种熵增强型策略评估方法，在智能体适应度评分中引入熵奖励，抑制种群行为趋同。该方法在保留原有帕累托优化框架的同时，注入行为空间多样性信号，促进多智能体环境中行为差异策略的探索。在具有显著不同奖励结构的火星车任务实验中，本方法相较NSGA-II基线，帕累托解集超体积提升最高达48%，表明行为多样性是提升多目标多智能体进化优化的重要且被低估的方向。
原文摘要 · Abstract (English)
Autonomous agent teams deployed in settings such as marine and extraterrestrial outposts must coordinate actions to achieve optimal outcomes across multiple competing objectives. Multi-objective evolutionary algorithms such as NSGA-II optimize for diversity in the objective space, but neglect diversity in the behavior space, possibly leading to premature convergence and a collapse in behaviors that may differentiate policies in different external conditions. To address this, we introduce an entropy-augmented policy evaluation strategy that incorporates an entropy bonus into agent fitness scores, discouraging behavioral homogeneity across the evolving population. By augmenting policy evaluation with a behavior-space diversity signal while preserving the underlying Pareto optimization framework, our method is designed to encourage exploration of behaviorally distinct policies in multiagent domains. We evaluate our approach across rover-domain experiments with qualitatively distinct reward structures and observe hypervolume improvements of up to 48% relative to the NSGA-II baseline, suggesting that behavioral diversity is a promising and underexplored direction for improving multi-objective multiagent evolutionary optimization.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。