不用近解态采样，直接从乱序魔方学解法。
Solving Rubik's Cube Without Tricky Sampling
- 用神经网络预测状态间代价模式，直接从乱序状态学习。
- 2×2×2魔方乱序5万次，解出率超99.4%。
- 仅靠策略网络成功求解，无需搜索，适合稀疏奖励问题。
魔方状态空间巨大且奖励稀疏，对强化学习构成重大挑战。以往研究通过从已解状态传播代价估计并结合搜索技术来缓解此问题，但这些方法与人类从完全混乱状态开始的解题方式不同。本文提出一种新型强化学习算法，采用策略梯度方法，在不依赖接近解态采样条件下解决魔方问题。该方法利用神经网络预测状态间的代价模式，使智能体能直接从乱序状态中学习。实验在2×2×2魔方上进行，共随机打乱50,000次，模型成功在超过99.4%的情况下完成求解。值得注意的是，该结果仅通过策略网络实现，未使用树搜索等复杂结构，验证了其在稀疏奖励任务中的有效性与可扩展性。
原文摘要 · Abstract (English)
The Rubiks Cube, with its vast state space and sparse reward structure, presents a significant challenge for reinforcement learning (RL) due to the difficulty of reaching rewarded states. Previous research addressed this by propagating cost-to-go estimates from the solved state and incorporating search techniques. These approaches differ from human strategies that start from fully scrambled cubes, which can be tricky for solving a general sparse-reward problem. In this paper, we introduce a novel RL algorithm using policy gradient methods to solve the Rubiks Cube without relying on near solved-state sampling. Our approach employs a neural network to predict cost patterns between states, allowing the agent to learn directly from scrambled states. Our method was tested on the 2x2x2 Rubiks Cube, where the cube was scrambled 50,000 times, and the model successfully solved it in over 99.4% of cases. Notably, this result was achieved using only the policy network without relying on tree search as in previous methods, demonstrating its effectiveness and potential for broader applications in sparse-reward problems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。