通过联合预测多智能体动作,提升组合优化的自改进效率与协同能力。
Multi-Action Self-Improvement for Neural Combinatorial Optimization
- 联合预测所有智能体的任务分配,避免逐个动作生成
- 使用集合预测损失,利用动作顺序对称性提升样本效率
- 并行生成多智能体动作,显著降低求解延迟
自改进已成为神经组合优化(NCO)的前沿范式,模型通过生成和模仿高质量解来迭代优化策略。然而,现有方法存在计算成本高、难以捕捉多智能体协同结构的问题。由于仅监督单动作轨迹,忽略智能体排列对称性,导致泛化能力差。本文将自改进扩展至联合多智能体动作空间,模型在每一步决策中联合预测完整任务分配。引入集合预测损失,对同一状态下的多个专家分配进行监督,显式利用对称性。该方法提升样本效率,增强协同行为学习能力。同时,通过并行生成多智能体动作,大幅加速自改进循环中的解生成阶段。在多个组合优化问题上验证,新方法在最终解质量与生成延迟方面均优于标准自改进。
原文摘要 · Abstract (English)
Self-improvement has emerged as a state-of-the-art paradigm in Neural Combinatorial Optimization (NCO), where models iteratively refine their policies by generating and imitating high-quality solutions. Despite strong empirical performance, existing methods face key limitations. Training is computationally expensive, as policy updates require sampling numerous candidate solutions per instance to extract a single expert trajectory. More fundamentally, these approaches fail to exploit the structure of combinatorial problems involving the coordination of multiple agents, such as vehicles in min-max routing or machines in scheduling. By supervising on single-action trajectories, they fail to exploit agent-permutation symmetries, where distinct sequences of actions yield identical solutions, hindering generalization and the ability to learn coordinated behavior. We address these challenges by extending self-improvement to operate over joint multi-agent actions. Our model architecture predicts complete agent-task assignments jointly at each decision step. To explicitly leverage symmetries, we employ a set-prediction loss, which supervises the policy on multiple expert assignments for any given state. This approach enhances sample efficiency and the model's ability to learn coordinated behavior. Furthermore, by generating multi-agent actions in parallel, it drastically accelerates the solution generation phase of the self-improvement loop. Empirically, we validate our method on several combinatorial problems, demonstrating consistent improvements in the quality of the final solution and a reduced generation latency compared to standard self-improvement.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。