人类如何快速找到近优解?研究发现是结构化学习+强化学习+搜索的结合。
Understanding Human-like Solutions in Combinatorial Optimization via Learning and Search

- 用指针网络模拟人类解法,通过最优解预训练+强化学习微调。
- 人类解虽非最优,但结构接近最优,且有系统性偏差。
- 最佳模型为最优预训练后强化学习微调,再用采样优化策略。
人类在时间与计算资源受限的情况下,仍能快速生成近似最优的欧几里得旅行商问题(TSP)路径。本研究通过大规模行为实验与计算分析,收集人类解并对比基于指针网络的神经策略。这些网络在多种目标下训练:强化学习(RL)、从最优解监督学习、从人类解监督学习,以及在最优监督预训练后的强化学习微调。结果显示,人类解并非与最优解完全一致,但位于近优几何区域:共享大量最优解的结构特征,同时保留特定的人类偏差。最佳模型是先在最优解上预训练,再经强化学习微调,并通过“最大-\text{N}”采样解码。该结果表明,人类式解法可能源于结构化监督学习、强化学习与测试时搜索的协同作用,呼应了现代人工智能系统的计算原理。
原文摘要 · Abstract (English)
Humans often find good solutions to combinatorial optimization problems that are computationally hard even for advanced computer algorithms. In the Euclidean traveling salesman problems (TSP), people rapidly produce tours that are near-optimal, despite severe limits on time and computation. What makes a tour human-like, and how might such solutions be learned? Here we address these questions through a large-scale behavioral and computational investigation of human performance in Euclidean TSP. We sampled a broad space of TSP instances, collected human solutions, and compared them with neural policies based on Pointer Networks, which are recurrent neural networks with an attention-based pointing mechanism that define probability distributions over valid tours. We trained these networks under multiple objectives, including reinforcement learning (RL), supervised learning from optimal tours, supervised learning from human tours, and RL fine-tuning after optimal-supervised pretraining. Human tours were not identical to optimal tours, but occupied a near-optimal geometric basin: they shared many structural properties with optimal solutions while preserving systematic human-specific deviations. The best account of human tours was not direct imitation of optimal tours, but a model pretrained on optimal tours, fine-tuned by RL, and decoded through $\text{Best-of-}N$ sampling. These findings suggest that human-like solutions may emerge from a combination of structured supervised learning, RL, and test-time search, echoing computational principles underlying many modern artificial intelligence systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。