混合强化学习与扩散模型,提升组合优化求解精度
HyCO: A Hybrid Neural Solver for Combinatorial Optimization

- 用强化学习构建初始解,再切换到扩散模型完成剩余决策
- 实验显示比单一方法平均提升12%以上,且误差增长更慢
- 自适应触发机制能有效捕捉策略转变时机,适合复杂优化任务
序列强化学习(RL)求解器和全局扩散模型(DM)求解器在组合优化中表现出互补的失败模式:前者在早期阶段后悔率小,但随步数呈超线性累积错误;后者避免了累积误差,但其后悔率与未解决子空间维度呈线性或亚线性关系。本文提出混合神经求解器HyCO,先用RL构建解的前缀,再自适应切换至条件扩散模型完成剩余决策。我们建立统一的误差缩放理论框架,证明在特定误差假设下,混合结构期望后悔率严格低于任一基线模型,且存在唯一最优切换步骤使总后悔最小。进一步设计轻量级自适应触发机制,结合策略熵与RL-DM分歧作为轨迹级转换信号的代理指标。实验证明,HyCO在多个基准上持续优于两种基线,并验证了自适应触发的有效性。
原文摘要 · Abstract (English)
Sequential reinforcement learning (RL) solvers and global diffusion model (DM) solvers for neural combinatorial optimization exhibit complementary failure modes under an optimization-regret view. The former enjoys small marginal regret in the early construction stage, but suffers from horizon-wise compounding errors with super-linear regret growth; the latter avoids horizon compounding but incurs linear or sublinear regret w.r.t. the dimension of the remaining unsolved subspace. We propose Hybrid Neural Solver for Combinatorial Optimization (HyCO), a hybrid inference algorithm that constructs a solution prefix with an RL solver and adaptively switches to a conditional DM to complete the remaining decisions. To characterize why such hybridization helps, when to trigger the handover, and how to realize it in practice, we first develop a unified error-scaling theoretical framework and prove that, under explicit error-scaling assumptions, i) the hybrid structure achieves strictly lower expected regret than either backbone alone, and ii) there exists a unique optimal trigger step that minimizes the hybrid regret. We then design a lightweight adaptive trigger that combines policy entropy and RL-DM disagreement to detect trajectory-level signals of the regime shift as a practical proxy, since the optimal trigger step is defined at the expected-regret level and is not directly computable on individual trajectories. Experimental results on diverse benchmarks demonstrate that HyCO achieves consistent improvements over both backbones and support the empirical effectiveness of adaptive triggering.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。